You are on page 1of 194

Automating Coded UI Testing

using Microsoft Testing Tools


in Visual Studio 2012

Introductions

Name
Title/Function
Job Responsibility
Testing Experience
C# Experience
Expectations for the Course
GSAtrain

Custom Coded UI Training for Microsoft

Course Materials

Lab Manual - Printed


Course Notes - PowerPoint Slides
Virtual Machines for TFS and VS 2012
Links and Whitepapers (Favorites)

GSAtrain

Custom Coded UI Training for Microsoft

Course Contents

1) Overview of Automated Code UI Testing


2) Walkthrough Steps from Beginning-to-End
3) Creating Manual Tests
4) Creating a Coded UI Test Using Coded UI Test
Builder
5) Automating Coded UI Test Cases
GSAtrain

Custom Coded UI Training for Microsoft

Course Contents
(Continued)
6) Data Driven Coded UI Tests
7) Anatomy of Code UI Tests
8) Introduction to Lifecycle Management
9) Extending Coded UI Tests and Action Recordings
10) Testing Large Applications with Multiple UI
Maps
GSAtrain

Custom Coded UI Training for Microsoft

Course Contents
(continued)
11) Using HTML5 Controls in Coded UI Tests
12) Visual Studio 2012 ALM Features
13) Upgrading Coded UI Tests from
Visual Studio 2010
14) Using WPF with Coded UI Tests
15) Best Practices for Coded UI Tests
GSAtrain

Custom Coded UI Training for Microsoft

Course Contents
(optional)
16) Using the Coded UI Test Logs
17) Using SharePoint with Coded UI Tests
18) ExcelCodedUIHelper

GSAtrain

Custom Coded UI Training for Microsoft

Module 1
Overview of Automated
Coded UI Testing

GSAtrain

Custom Coded UI Training for Microsoft

Automated Code UI Testing

UI Testing
o Functional Testing

Controls
User Interface
Whole Application

The earlier we catch a bug or potential


problem the less it costs!
o Image
o Manpower
GSAtrain

Custom Coded UI Training for Microsoft

Reality of Automated
Testing
70% of all Testing is manual

GSAtrain

10

Custom Coded UI Training for Microsoft

What is Coded UI Testing?

Replacing a Manual User Interface Test with an


Automated Test.
o To Allow:
Replication
Consistency
Regression Testing For:
Multiple Environments
Multiple Versions
GSAtrain

11

Custom Coded UI Training for Microsoft

IT Has a Problem

GSAtrain

12

Custom Coded UI Training for Microsoft

Test Early and Often

Catching defects as early as possible is the least


expensive way to ensure software quality
Kent Beck and Cynthia Andres wrote
o "Here is the dilemma in software development:
defects are expensive, but eliminating defects is also
expensive.
o However, most defects end up costing more than it
would have cost to prevent them."
GSAtrain

13

Custom Coded UI Training for Microsoft

The Testing Trends


Forwards
Test-Driven Development
o Design the Test
o Then Write the Code

GSAtrain

14

Custom Coded UI Training for Microsoft

The Testing Trends:


Backwards and Circular

Linking it all together


o Help-Desk
o Test
o Story Board
o Task
o Unit
o Programmer
GSAtrain

15

Custom Coded UI Training for Microsoft

The Ultimate Goal


of Software Testing
To find the bugs before your users
do!
o UI Testing is END USER TESTING
o OR TOTAL SYSTEM TESTING
o OR INFRASTRUCTURE TESTING

GSAtrain

16

Custom Coded UI Training for Microsoft

Manual to Coded UI

GSAtrain

17

Custom Coded UI Training for Microsoft

Testers Multiple Roles

Guarantee SOFTWARE QUALITY


Regression Test each Version or Build
Test all Environments
Provide Developers with all the information required:
o To REPRODUCE THE BUG
o Trace the steps to get the BUG
o Screen shots, IntelliTrace, Logs and Stack Traces

NO-REPRO must be ELIMINATED


GSAtrain

18

Custom Coded UI Training for Microsoft

Tiers of Software Testing

Clear Separation of Functions


Clear Definition of Communication and
Collaboration
TEAM of Software Quality

GSAtrain

19

Custom Coded UI Training for Microsoft

Module 2
Walkthrough Steps
from Beginning-to-End

GSAtrain

20

Custom Coded UI Training for Microsoft

Walkthrough Steps from


Beginning-to-End
Lesson 1 Manual Testing
o Using Microsoft Test Manger to create and manage Test Plans
o Running Test Cases and Publishing Results
o Conducting Exploratory Testing
o Fast Forward for Manual Testing

Lesson 2 Coded User Interface Testing


o Automated Functional Tests Using CUIT
o Create a CUIT from Scratch
o Create a CUIT from action recording
o Making CUIT more Robust and Flexible
GSAtrain

21

Custom Coded UI Training for Microsoft

A Manual Testing Process

Plan

GSAtrain

Design

Test

Track

22

Verify

Autom
ate

Custom Coded UI Training for Microsoft

Building an App
For Manual Testing
Demo of Simple Calculator

GSAtrain

23

Custom Coded UI Training for Microsoft

Automating the Calculator

Demo

GSAtrain

24

Custom Coded UI Training for Microsoft

Adding Data

Demo

GSAtrain

25

Custom Coded UI Training for Microsoft

Introducing an Error

Demo

GSAtrain

26

Custom Coded UI Training for Microsoft

Hands-On Lab

Walkthrough Steps from Beginning-to-End


o Examine the code to be tested
o Create a Coded UI
o Test the Coded UI
o Add a data file to drive the UI Tests
o Run all of the tests
o Create errors in the data file
o Examine failed tests
GSAtrain

27

Custom Coded UI Training for Microsoft

Module 3
Creating Manual Tests

GSAtrain

28

Custom Coded UI Training for Microsoft

Visual Studio 2012


Supported Test Types

GSAtrain

29

Custom Coded UI Training for Microsoft

Microsoft Test Manager

Provides a single environment to author


and manage Tests

GSAtrain

30

Custom Coded UI Training for Microsoft

Testing Center

GSAtrain

31

Custom Coded UI Training for Microsoft

Generate Code for Coded UI


Test

GSAtrain

32

Custom Coded UI Training for Microsoft

Quick Start Guide for Manual Testing


using Microsoft Test Manager

http://msdn.microsoft.com/en-us/library/
dd380763.aspx

GSAtrain

33

Custom Coded UI Training for Microsoft

Easily Reproducing Issues


Through Manual Testing
Introducing Managed Exploratory
Testing
The 5 Minute elevator Video

GSAtrain

o http
://channel9.msdn.com/Series/Visual-Studio-2
012-Premium-and-Ultimate-Overview/Visual-St
udio-Ultimate-2012-Easily-reproducing-issue
s-through-manual-testing
34

Custom Coded UI Training for Microsoft

Module 4
Creating a Coded UI Test
Using Coded UI Test Builder

GSAtrain

35

Custom Coded UI Training for Microsoft

Creating a Coded UI Test


Using Coded UI Test Builder
Test Types
Microsoft Test Manager
Exploratory Testing
Easily reproducing issues through
manual testing

GSAtrain

36

Custom Coded UI Training for Microsoft

Microsoft Test Manager

The first time you run MTM it will prompt


o Add Team Foundation Manager
If you dont know the Name, contact your
administrator
It might require ServerName:Port Number

GSAtrain

37

Custom Coded UI Training for Microsoft

TFS and MTM

GSAtrain

38

Custom Coded UI Training for Microsoft

Connecting Microsoft Test


Manager to TFS
MTM is always connected to TFS

GSAtrain

39

Custom Coded UI Training for Microsoft

Test Plan Properties

GSAtrain

40

Custom Coded UI Training for Microsoft

MTM Test Suites

Static suite: the content of this suite is


manually added test cases
Query-based suite: a query-based suite lists all
test cases matching a given work item filter
Requirements-based suite: this suite shows the
test cases associated with a selected TFS
requirement
GSAtrain

41

Custom Coded UI Training for Microsoft

Custom Coded UI Training for


Microsoft

GSAtrain

42

Creating a Lab Environment

With Visual Studio 2012 Lab Management it is


possible to create an environment based on
existing infrastructure without SCVMM
o This is great because we can now start using Lab
Management without any additional installation
required
o The lab machines in a standard environment can be
physical or virtual (or both) and the virtual machine
does not have to be managed by SCVMM or Hyper-V
GSAtrain

43

Custom Coded UI Training for Microsoft

Creating a Lab Environment


(continued)
To create a standard environment you only need to
know the names of the machine you want to add to
the environment and a user account to be used to log
in and deploy the test agent service onto the
machine
A standard environment can be connected to directly
from MTM and the Test Runner making the
integration with lab environments really smooth
GSAtrain

44

Custom Coded UI Training for Microsoft

Creating a Lab Environment


(continued)
The alternative to standard environments is to use a
SCVMM environment
The key features of a SCVMM environment include
o Create new machines from templates
New lab machines can automatically be created by anyone
(having permissions) running MTM
It is no longer required to send a request to the operations
department to get machines created for a new environment or
to do other maintenance operations such as starting, stopping,
snapshotting, or restoring machines
GSAtrain

45

Custom Coded UI Training for Microsoft

Creating a Lab Environment


(continued)
Support for snapshots
o It is possible to work with snapshots from MTM,
which enable testers to save state during testing
o This is a powerful feature both to unlock the
tester if a test case fails (no need to block an
environment while waiting for a resolution) but
also to give a developer a fixed point in time to
reproduce a bug
GSAtrain

46

Custom Coded UI Training for Microsoft

Creating a Lab Environment


(continued)
Connection inside MTM
o A SCVMM environment can be connected to
MTM and the Test Runner just like with the
standard environment

GSAtrain

47

Custom Coded UI Training for Microsoft

KPIs For Software Testing

Number of bugs per state


o How many bugs do we have that are active, resolved, or closed?
o Are the number of active bugs increasing and number of
resolved and closed bugs constant?
o Then we need to look into how we perform our testing

Number of bugs sent back from testers for more


information
o A large number might indicate that communication between
developers and testers must improve
GSAtrain

48

Custom Coded UI Training for Microsoft

KPIs For Software Testing


(continued)
Code coverage
o This shows us how much of the code has been covered
by our automated unit tests
o We get the value as a percentage of the whole code

Tests run results


o How are our tests performing?
o Do we have many failed tests?
o If so, what can we do to improve this?
GSAtrain

49

Custom Coded UI Training for Microsoft

KPIs For Software Testing


(continued)
Percent Requirements covered by test cases
o Do we write test cases for all our requirements?
o If not, what is the reason?

Percent Requirements covered by testing


o Do we actually run the tests we have test cases for?
o If this figure is low and the figure for Percent
Requirements covered by test cases is high we might
have an issue we need to deal with
GSAtrain

50

Custom Coded UI Training for Microsoft

Application Tier:
What Does TFS Offer?
First we find all web services for team project
collections
We all recognize this so no further explanation is
necessary:
o Version control
o Build service
o Lab management
o Work Item tracking
o Team Foundation Framework services
GSAtrain

51

Custom Coded UI Training for Microsoft

Application Tier:
What Does TFS Offer?
(continued)

This set of services is running on the


Application Tier of TFS and allows us to access
the different aspects of Team Foundation Server,
including administration, security, and events
They enable us to extend Team Foundation
functionality by developing TFS extensions and
adapters to integrate third-party tools with TFS
2012
GSAtrain

52

Custom Coded UI Training for Microsoft

Custom Coded UI Training for


Microsoft

GSAtrain

53

Custom Coded UI Training for


Microsoft

GSAtrain

54

Custom Coded UI Training for


Microsoft

GSAtrain

55

Testers Benefits

After you have run your tests, you can report on your
progress
When you use test plans to structure your testing
approach, you can answer the following questions:
o How many tests have passed or failed?
o How many tests still have to be run for this iteration?
o When will the testing be complete?
o Which areas of the product have high test failure rates?
GSAtrain

56

Custom Coded UI Training for Microsoft

Testers Benefits
(continued)
o Which test configurations have high test
failure rates?
o Who has the most tests left to be run?
o Can resources be reallocated to better balance
the remaining testing?
o Which build should the testers be using
based on code changes and bug fixes?
GSAtrain

57

Custom Coded UI Training for Microsoft

Essential Guide for Running


Automated Tests from a Test Plan

GSAtrain

58

Custom Coded UI Training for Microsoft

Creating a CUIT

You perform the test manually while CUIT


Test Builder runs in the Background
You can specify the Values that should
appear in specific fields
The CUIT Test Builder Records actions and
generates Code
Your Main Skills are in Testing not Coding
GSAtrain

59

Custom Coded UI Training for Microsoft

If You Are a Developer

You can extend the CUIT code


The CUIT Test Builder builds Structured
Code
Straightforward to copy and adapt

GSAtrain

60

Custom Coded UI Training for Microsoft

Required Software

Visual Studio Ultimate


or
Visual Studio Premium

GSAtrain

61

Custom Coded UI Training for Microsoft

The 3 Major UI Test Types

Planned
Exploratory
Load and Muli-user

GSAtrain

62

Custom Coded UI Training for Microsoft

Defining Your Testing Effort


Using Test Plans
Microsoft Test Manager enables you to
do three important tasks, as shown in
the following illustration:

GSAtrain

63

Custom Coded UI Training for Microsoft

Exploratory Testing

Exploratory testing seeks to find out how the


software actually works, and to ask questions about
how it will handle difficult and easy cases
The quality of the testing is dependent on the
tester's skill of inventing test cases and finding
defects
The more the tester knows about the product and
different test methods, the better the testing will be
GSAtrain

64

Custom Coded UI Training for Microsoft

Benefits of
Exploratory Testing
The main advantage of exploratory testing is that
less preparation is needed, important bugs are
found quickly, and at execution time, the approach
tends to be more intellectually stimulating than
execution of scripted tests
Another major benefit is that testers can use
deductive reasoning based on the results of previous
results to guide their future testing on the fly
GSAtrain

65

Custom Coded UI Training for Microsoft

Benefits of
Exploratory Testing
(continued)
Another benefit is that, after initial testing, most
bugs are discovered by some sort of exploratory
testing
This can be demonstrated logically by stating,
"Programs that pass certain tests tend to continue to
pass the same tests and are more likely to fail other
tests or scenarios that are yet to be explored
WikiPedia
GSAtrain

66

Custom Coded UI Training for Microsoft

DrawBacks of
Exploratory Testing
Disadvantages are that tests invented and performed on the fly
can't be reviewed in advance (and by that prevent errors in code
and test cases), and that it can be difficult to show exactly which
tests have been run
Freestyle exploratory test ideas, when revisited, are unlikely to
be performed in exactly the same manner, which can be an
advantage if it is important to find new errors; or a disadvantage
if it is more important to repeat specific details of the earlier tests
o Can be controlled with specific instruction to the tester, or by preparing
automated tests where feasible, appropriate, and necessary, and ideally
as close to the unit level as possible
GSAtrain

67

Custom Coded UI Training for Microsoft

Exploratory Testing

Exploratory testing is particularly


suitable if requirements and
specifications are incomplete, or if there
is lack of time
Same approach can also be used to verify
that previous testing has found the most
important defects
GSAtrain

68

Custom Coded UI Training for Microsoft

Custom Coded UI Training for


Microsoft

GSAtrain

69

Visual Studio
and Exploratory Testing
Performing Exploratory Testing Using Microsoft
Test Manager
o In Microsoft Test Manager:
Open Testing Center
Test
Do Exploratory Testing

o If you don't see the option to Do Exploratory Testing,


then your team is using a version of TFS prior to 2012
GSAtrain

70

Custom Coded UI Training for Microsoft

Coded User Interface


Testing
Automated Functional Tests Using
CUIT
Create a CUIT from Scratch
Create a CUIT from action recording
Making CUIT more Robust and Flexible

GSAtrain

71

Custom Coded UI Training for Microsoft

Contents of a Coded UI Test

When you create a Coded UI Test, the


Coded UI Test Builder creates a map of
the user interface under test, and also the
test methods, parameters, and assertions
for all tests
It also creates a class file for each test
GSAtrain

72

Custom Coded UI Training for Microsoft

Testing - What is New in VS


2012?
Conduct exploratory test sessions
Locate, view, and edit your test methods, UI
actions and their associated controls in the UI
control map
Quickly track your test plan progress using the
Results tab in Microsoft Test Manager
Clone your test plan to work on different releases
in parallel
GSAtrain

73

Custom Coded UI Training for Microsoft

Testing - What is New in VS 2012?


(continued)

Test Windows Store apps running on a remote


device using manual tests or exploratory test
sessions
Install and configure agents more easily for Lab
Management for Visual Studio 2012
Include SCVMM 2012 and clustered Hyper-V
hosts in Lab Management for Visual Studio 2012
GSAtrain

74

Custom Coded UI Training for Microsoft

Contents Test
Configurations

GSAtrain

75

Custom Coded UI Training for Microsoft

Configuring Test Settings

Establish Environments
Diagnostic Connectors
Best Practice
o Work with Development Teams
o The Developers need to have the same tools to view the test
data you collected for them

Microsoft released a Test Attachment Cleaner available


at
o Aka.ms./TFPowerTools
GSAtrain

76

Custom Coded UI Training for Microsoft

Using Builds

Scout
o Installing Software and running some initial tests
o After the SCOUT is successful it can become a
template for other Testers

There is an available builds DROPDOWN


Click ASSIGN TO PLAN to add a new
build to a plan
GSAtrain

77

Custom Coded UI Training for Microsoft

Analyzing Impacted Tests

Quickly identify tests that need to be run based on


changes to code
Can Run in the background
Track menu => Recommended Tests
o Does not recognize DATA changes
o Does not recognize other library changes
o Does not recognize other Application changes

Best Practice - Periodically run all Tests


GSAtrain

78

Custom Coded UI Training for Microsoft

Defining Multiple
Test Configurations
Environments may differ:
o IE 8,9,10
o Ipad Horizontal/Vertical
o BYOD
o HTML5
o
GSAtrain

Etc.
79

Custom Coded UI Training for Microsoft

Module 5
Automating
Coded UI Test Cases

GSAtrain

80

Custom Coded UI Training for Microsoft

Test Cases Are


Organized Into Suites
Requirements-Based test suite
Query-Based test suite
Static Test Suite
o Status of tests in TEST SUITE
In planning
In Progress
Completed
GSAtrain

81

Custom Coded UI Training for Microsoft

Types of Tests

Unit and Component Tests


Functional Tests
Exploratory Testing
Capability Testing
GSAtrain

83

Custom Coded UI Training for Microsoft

Unit and Component Tests

Automated tests written to help the team develop software


effectively
With good suites of unit and component level tests we have
the safety net that helps us develop software incrementally
in short iterations without breaking existing functionality
Invaluable when refactoring code
o With good test coverage a developer should feel confident to make
a change without knowing all about every dependency
o The tests should tell us if we did anything wrong
GSAtrain

84

Custom Coded UI Training for Microsoft

Functional Tests

Functional tests are mainly our traditional


scripted system tests in different flavors
It is hard to avoid running these tests manually at
first, but we should try to find ways to automate
them as we learn more about our product and
how it needs to be tested
Many functional tests can be automated and then
we can focus on early testing for the manual tester
GSAtrain

85

Custom Coded UI Training for Microsoft

Exploratory Testing

Exploratory testing is a form of software testing in


which the individual tester can design and run tests in
a freer form
Instead of following detailed test scripts, the tester
explores the system under tests based on the user
stories
As the tester learns how the system behaves the tester
can optimize the testing work and focus more on
testing than documenting the test process
GSAtrain

86

Custom Coded UI Training for Microsoft

Capability Testing

Lastly we have the capability tests


These tests are run against the behavior of the system;
o We test non-functional requirements
Performance
Security

These tests are generally automated and run using


special purpose tools, such as load test frameworks
and security analyzers
GSAtrain

87

Custom Coded UI Training for Microsoft

Exploratory Testing

We should leave this category of tests as manual


tests
The focus should be to catch bugs that would fall
through the net of automated tests
A key motivation for automated testing is to let
do more of exploratory and usability testing
because these tests validate how the end-user
feels when using the product
GSAtrain

88

Custom Coded UI Training for Microsoft

Custom Coded UI Training for


Microsoft

GSAtrain

89

Story Boards

A user story can be stated as simply as


o As an Employee I want to have an efficient way to
manage my expenses

The conversation around this statement


between the product owner and the
development team can raise questions such as:
o Who can submit expense reports?
GSAtrain

90

Custom Coded UI Training for Microsoft

Story Boards
(continued)
o What states can an expense report be in?
o When is it possible to change or remove an
expense report?
o What data is required in an expense report to
register it correctly?
o Where are the expense reports stored? For how
long?
GSAtrain

91

Custom Coded UI Training for Microsoft

Adding Detail to User


Stories
An expense report has the following state
model:
o New when created
o Pending after submitted for approval
o Approved
o Rejected
GSAtrain

92

Custom Coded UI Training for Microsoft

Story Boards Are


Vital to Testing
Allows the Tester to Verify what should happen
If this does not Happen
o See the Tasks that were assigned to this requirement
o See the Programmers assigned to Each Task
o Notify the Developer(s)
Provide the necessary info to avoid NO REPRO

Add this Test to the Suite


GSAtrain

93

Custom Coded UI Training for Microsoft

Hands-On Lab

Introduction to Test Planning with


Microsoft Test Manager 2012
o Authoring Test Cases

GSAtrain

94

Custom Coded UI Training for Microsoft

Module 6
Data Driven Tests

GSAtrain

95

Custom Coded UI Training for Microsoft

Data Driven Tests

Any Files Used in The Tests


Example XML to load Variable and Expected Data
Could be:
o SQL
o Business Connectivity Services
o SharePoint Lists
o SQL Reports
o Excel / PowerPivot
o SSAS
GSAtrain

96

Custom Coded UI Training for Microsoft

Uses of Data Driven Tests

Create errors
Add new Cases
Modify the rows
Automation can be changed by using
different files

GSAtrain

97

Custom Coded UI Training for Microsoft

Hands-On Lab

How to Create
a Data-Driven Coded UI Test

GSAtrain

98

Custom Coded UI Training for Microsoft

The UNFORTUNATE REALITY

70% of all Application Testing is


MANUAL

GSAtrain

99

Custom Coded UI Training for Microsoft

Module 7
Anatomy of
Coded UI Tests

GSAtrain

100

Custom Coded UI Training for Microsoft

Anatomy of Coded UI Tests

When you create a Coded UI Test, the Coded


UI Test Builder creates a map of the user
interface under test, and also the test methods,
parameters, and assertions for all tests
It also creates a class file for each test
o http
://msdn.microsoft.com/en-us/library/ff398062.aspx
GSAtrain

101

Custom Coded UI Training for Microsoft

Contents of a Coded UI Test

GSAtrain

File

Contents Editable?

UIMap.Designer.cs

Declarations
section
UIMap class
(partial, autogenerated)
Methods
Properties

No

UIMap.cs

UIMap class
(partial)

Yes

CodedUITest1.cs

CodedUITest1 class
Methods
Properties

Yes

102

Custom Coded UI Training for Microsoft

Contents of Coded UI Test


(continued)
UIMap.Designer.cs
o This file contains code that is automatically
created by the Coded UI Test Builder when a
test is created
o This file is re-created every time that a test
changes, so that it is not a file in which you
can add or modify code
GSAtrain

103

Custom Coded UI Training for Microsoft

Declarations section
using System;
using System.CodeDom.Compiler;
using System.Collections.Generic;
using System.Drawing;
using System.Text.RegularExpressions;
using System.Windows.Input;
using Microsoft.VisualStudio.TestTools.UITest.Extension;
using Microsoft.VisualStudio.TestTools.UITesting;
using Microsoft.VisualStudio.TestTools.UITesting.WinControls;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using Keyboard =
Microsoft.VisualStudio.TestTools.UITesting.Keyboard;
using Mouse = Microsoft.VisualStudio.TestTools.UITesting.Mouse;
using MouseButtons = System.Windows.Forms.MouseButtons
GSAtrain

104

Custom Coded UI Training for Microsoft

Module 8
Introduction to
Lifecycle Management

GSAtrain

105

Custom Coded UI Training for Microsoft

Visual Studio 2012

Visual Studio Ultimate, Visual Studio


Premium and Test Professional include a
Microsoft Test Manager
o To help you define and manage your testing
effort by using test plans
o You create a test plan and add any test suites,
test cases, or configurations that you need
GSAtrain

106

Custom Coded UI Training for Microsoft

Lifecycle Management

By using the suite of tools in Visual Studio 2012, and


combining those tools with Visual Studio Team
Foundation Server, you can apply proven practices to
manage your application's lifecycle, from
understanding customer needs through code design
and implementation to deployment
You can use the instrumentation in these tools to
trace requirements to checked-in code, builds and
test results
GSAtrain

107

Custom Coded UI Training for Microsoft

Lifecycle Management
(continued)
These practices can help your team create software that is
valued by your customers and that is faster and more reliable.
You can use these tools to achieve the following results:
o Visual Studio Application Lifecycle Management (ALM) provides
you and your team with the tools to effectively manage the lifecycle
to develop applications
o As the following illustration shows, you and your team can use these
tools to tell user stories, to manage the backlog, plan a sprint, and
run a sprint, to engage stakeholders to provide continuous feedback,
and to easily incorporate feedback to support the next iteration of
software
GSAtrain

108

Custom Coded UI Training for Microsoft

ALM Cycles

GSAtrain

109

Custom Coded UI Training for Microsoft

What's New for ALM


in Visual Studio 2012?
http://msdn.microsoft.com/enus/library/bb385832.aspx

GSAtrain

110

Custom Coded UI Training for Microsoft

Visual Studio 2010 to 2012:


Completely Restructured Software Testing

In 2012 Visual Studio Improves:


o Integration between testing and the rest of the
Software Development lifecycle
o Example Better handoff from tester to developer
o Less NO REPRO
o Improved build-deploy-test workflow
o Improved Virtualization
GSAtrain

111

Custom Coded UI Training for Microsoft

Types of Tests
Supported by VS 2012
Manual Tests
Coded UI Tests
Unit Tests
Web Performance Tests
Load Tests
Generic Tests
Ordered Tests
GSAtrain

112

Custom Coded UI Training for Microsoft

Testing Pyramid

GSAtrain

113

Custom Coded UI Training for Microsoft

Testing Structures

GSAtrain

114

Custom Coded UI Training for Microsoft

Using Shared Steps

Demo
LAB

GSAtrain

115

Custom Coded UI Training for Microsoft

Assigning Configurations

Demo

GSAtrain

116

Custom Coded UI Training for Microsoft

Manual Tests

Require a Human to Interact with an


application
Verify some expected result
Report on the success of the test

GSAtrain

117

Custom Coded UI Training for Microsoft

Coded UI Tests

Provides the Capability to author tests that


automatically interact with user interface
o Verify Results
o File Bug Report
o It can be run Frequently without human
intervention
o Typically more expensive to author and maintain
than manual test
GSAtrain

118

Custom Coded UI Training for Microsoft

Unit Tests

Used by Programmers for Programmers


Quickly becoming the standard practice
A vital part of Test-Driven Development
Generally focus on a lower level than other
Testing
Ideally a unit is the smallest atomic part - One
method of One object
GSAtrain

119

Custom Coded UI Training for Microsoft

Unit Tests
(continued)
Test written before the object
It will initially fail
o Correct
o Re-test
o Refactor
o Repeat

Essential component of Regression Testing


GSAtrain

120

Custom Coded UI Training for Microsoft

Web Performance Tests

Verify that the web site is behaving as


expected
o Add load to see if the application is still
capable of Scaling

GSAtrain

121

Custom Coded UI Training for Microsoft

Load Tests

Tests that Application Scales and


Performs as Necessary

GSAtrain

122

Custom Coded UI Training for Microsoft

Generic Tests

Calling external testing systems


Results can be automatically parsed to
determine success
Could be simple console application to
XML document exported from LOB

GSAtrain

123

Custom Coded UI Training for Microsoft

Ordered Tests

Container for other tests


Specific order of tests
Some run more than once

GSAtrain

124

Custom Coded UI Training for Microsoft

Diagnostic Data Adapters

The Key challenge of Testers


Provide Developers with sufficient
information to reproduce the problem
Most Bugs result in No Repro
It Works on my machine

GSAtrain

125

Custom Coded UI Training for Microsoft

Diagnostic Data Adapters

A Diagnostic Data Collector is responsible for collecting data about


the machine under test
o Action Log
o ASP.NET Profiler
o Code Coverage
o IntelliTrace
o Event Log
o System Information
o Test Impact
o Video Recorder
o Network Emulation
GSAtrain

126

Custom Coded UI Training for Microsoft

Developers view

Test-Driven Development
Unit Testing

GSAtrain

127

Custom Coded UI Training for Microsoft

Test-driven development
(TDD)
WikiPedia
Is a software development process that relies on
the repetition of a very short development cycle:
o First the developer writes an (initially failing)
automated test case
That defines a desired improvement or new function
Then produces the minimum amount of code to
pass that test
Refactors the new code to acceptable standards
GSAtrain

128

Custom Coded UI Training for Microsoft

Testers View

Not Unit Testing


Not Integration Testing
Not Regression Testing

Whole Application Testing

GSAtrain

129

Custom Coded UI Training for Microsoft

Requirements-Driven Testing

When requirements or user stories, or


features are ready to be tested, you can
run your tests for each configuration that
you specified
UI User Interface Testing
Usability - Testing
GSAtrain

130

Custom Coded UI Training for Microsoft

Components of
Coded UI Testing
Test Runner:
o The Test Runner from MTM can create action recordings from a
manual test
o The action recording can later be used to play back the test session
in MTM or to generate a Coded UI test from the recorded steps

Coded UI Test:
o The implementation of the actual test written in C# or VB.NET
o The Coded UI Test is either generated by the Coded UI Test
builder code generator or coded directly against the Coded UI Test
API
GSAtrain

131

Custom Coded UI Training for Microsoft

Components of
Coded UI Testing (continued)
Test Recorder:
Records actions from a test session, which then
can be used by the Test Runner to automate a
generated test or by the coded generator to create
a Coded UI test
Coded UI Test API:
The Coded UI Test API can be used to create UI tests
which are more tightly connected to the UI elements
GSAtrain

132

Custom Coded UI Training for Microsoft

Components of
Coded UI Testing (continued)
Technology Managers:
These are the core components responsible
for mapping UI elements to the underlying
technology (i.e. web browser or a rich client)
We can implement custom plug-ins to
support platforms which the native plug-ins
does not support
GSAtrain

133

Custom Coded UI Training for Microsoft

Custom Coded UI Training for


Microsoft

GSAtrain

134

Managing Automated Tests

These tests are critical to project success


The number keeps growing
Regression testing is vital
Sometimes hundreds or even thousand of
tests

GSAtrain

135

Custom Coded UI Training for Microsoft

Supported Platforms

Internet Explorer 8, 9, 10
Windows Forms 2.0 and later
WPF 3.5 and later
Windows Win32 (not officially supported but may work)
MFC (partially)
SharePoint (partially)
Dynamics CRM
Citrix/Terminal Services (partially)
PowerBuilder (partially)
GSAtrain

136

Custom Coded UI Training for Microsoft

Coded UI Test Builder


Interface

GSAtrain

137

Custom Coded UI Training for Microsoft

Parameterized Testing

A common requirement of automated tests is to be able to


run the test a number of times with different parameters
o We have already looked at how we can work with data-driven
test in details
o The same techniques are valid for coded UI tests as well except
that the data source for a UI test often points to the data set
managed in the associated test case
o Using the test case to manage test data for the automated test is
nice because then the tester can update the data directly from
inside MTM
GSAtrain

138

Custom Coded UI Training for Microsoft

Regression Testing

Coded UI tests are probably the ideal type of test for


regression testing because they give us the opportunity to
automatically test the system just like a real user would
But there are a number of issues that make UI tests difficult
as a general purpose solution for automation solution,
including
o Requires a UI
o Requires a desktop session to run
o UI tests are more complex to design and maintain
o Difficult to scale
GSAtrain

139

Custom Coded UI Training for Microsoft

Under-the-UI Testing

To work around these challenges we


generally recommend considering underthe-UI tests as a complement to the UI tests
These tests could be written against the
APIs in the system or against the controller
layer in an MVC style of application

GSAtrain

140

Custom Coded UI Training for Microsoft

Test Case Management


Command-Line Tool
Another option for running tests from the command-line is to use the
Test Case Management command-line tool, TCM.exe
o This tool is installed under
C:\Program Files (x86)\Microsoft Visual Studio
11.0\Common7\IDE\TCM.exe

There are several options for us to start a test run, for instance:
o Run all tests in a test suite
o Locate the Suite Id from the test suite in MTM
tcm run /create /title:"UI Test Run" /planid:5
/suiteid:11 /configid:2
/collection:http://tfs.kangreen.com:8080/tfs
/teamproject:"Expense Reporting"
GSAtrain

141

Custom Coded UI Training for Microsoft

Test Case Management


Command-Line Tool (continued)

You can use tcm.exe to figure out the Ids for plans, suites,
and configurations as well:
o List plans
tcm plans /list /collection:http://tfs.kangreen.com:8080/tfs
/teamproject:"Expense Reporting"

o List suites
tcm suites /list /collection:http://tfs.kangreen.com:8080/tfs
/teamproject:"Expense Reporting"

o List configurations
tcm configs /list /collection:http://tfs.kangreen.com:8080/tfs
/teamproject:"Expense Reporting"

GSAtrain

142

Custom Coded UI Training for Microsoft

Demo Creating a Test


Project

http://channel9.msdn.com/Series/VisualStudio-2012-Premium-and-Ultimate-Overvi
ew/Visual-Studio-Ultimate-2012-Easily-t
esting-user-interfaces-with-Coded-UI-te
sts

GSAtrain

143

Custom Coded UI Training for Microsoft

Components of a Test Plan

GSAtrain

144

Custom Coded UI Training for Microsoft

Benefits of Automated
Testing
Automated tests can quickly test whether
your application is still working correctly
after code changes have been made to
your application
Automating Test should not take more
than about 40% of team members' time
GSAtrain

145

Custom Coded UI Training for Microsoft

The Achilles Heel


of Automated Tests
The problem with user interface tests is not in how to
execute them, but what to test
Since human testers would often check the whole
round-trip of information, from the GUI to the backend report
First instinct is to replicate that and try to verify
business domain rules. This is the Achilles heel of UI
tests, and it ultimately leads to enormous waste of
time
GSAtrain

146

Custom Coded UI Training for Microsoft

UI Testing Without Shooting


Yourself in the Foot
http://gojko.net/2010/04/13/how-toimplement-ui-testing-without-shootingyourself-in-the-foot-2/

GSAtrain

147

Custom Coded UI Training for Microsoft

Thinking About
These Different Levels
The idea of thinking about these
different levels is good because it allows
us to write
o UI-level tests that are easy to understand
o Efficient to write
o Relatively inexpensive to maintain
GSAtrain

148

Custom Coded UI Training for Microsoft

Thinking About
These Different Levels (continued)

Checking that delivery is available for two books


involves putting a book in a shopping cart
o Putting a book in a shopping cart involves a sequence
of technical steps
o Entering address details does as well

Breaking things down like that and combining


lower level concepts into higher level concepts
reduces the cognitive load and promotes reuse
GSAtrain

149

Custom Coded UI Training for Microsoft

Using Microsoft Test


Manager
You can plan, manage and execute both manual and exploratory
tests
You can also automate your manual tests after they are stabilized
While you are performing manual and exploratory tests, you can
log bugs
o The bug will contain a trace of your recent actions, a snapshot of the
state of the system, and a copy of any notes you made while exploring
the system
o You can record your actions in the test case, so that they can be played
back on later occasions

GSAtrain

150

Custom Coded UI Training for Microsoft

Manual to Coded UI

GSAtrain

151

Custom Coded UI Training for Microsoft

Module 9
Extending Coded UI
Tests and Action
Recordings

GSAtrain

152

Custom Coded UI Training for Microsoft

Extending Coded UI Tests


and Action Recordings
Supported Configurations
o http://msdn.microsoft.com/enus/library/dd380742.aspx

GSAtrain

153

Custom Coded UI Training for Microsoft

Module 10
Upgrading Coded UI Tests from
Visual Studio 2010

GSAtrain

154

Custom Coded UI Training for Microsoft

Module 11
Extending Coded UI Tests
and Action Recordings

GSAtrain

155

Custom Coded UI Training for Microsoft

Extending Coded UI Tests and Action


Recordings to Support Microsoft Excel

The testing framework for coded UI tests and action


recordings does not support every possible user interface
o It might not support the specific UI that you want to test
o For example, you cannot immediately create a coded UI test or
an action recording for a Microsoft Excel spreadsheet

However, you can create your own extension to the coded


UI test framework that will support your specific UI by
taking advantage of the extensibility of the coded UI test
framework
GSAtrain

156

Custom Coded UI Training for Microsoft

Extending Coded UI Tests and Action


Recordings to Support Microsoft Excel
(continued)

The following topic gives an example of how to extend the


framework to support the creation of coded UI tests and
action recordings for Microsoft Excel
o For more information about the platforms that are supported, see
Supported Configurations and Platforms for Coded UI Tests and
Action Recordings

This section presents a coded UI test extension that can


record and play back tests of Excel Worksheets
o Each part of the extension is explained in this section and in the
code comments for developers who want to create just such an
extension
GSAtrain

157

Custom Coded UI Training for Microsoft

Architecture Overview

GSAtrain

158

Custom Coded UI Training for Microsoft

Module 12
Testing Large Applications
with Multiple UI Maps

GSAtrain

159

Custom Coded UI Training for Microsoft

Module 13
Using HTML5 Controls
in Coded UI Tests

GSAtrain

160

Custom Coded UI Training for Microsoft

Module 14
Using WPF with
Coded UI Tests

GSAtrain

161

Custom Coded UI Training for Microsoft

Module 15
Best Practices
for Coded UI Tests

GSAtrain

162

Custom Coded UI Training for Microsoft

Best Practices
For Coded UI Tests

Use the Coded UI Test Builder whenever possible

Do not modify the UIMap.designer.cs file directly. If you do this,


the changes to the file will be overwritten

Create your test as a sequence of recorded methods. For more


information about how to record a method, see Creating Coded UI
Tests

Each recorded method should act on a single page, form, or dialog


box

Create a new test method for each new page, form, or dialog box

GSAtrain

163

Custom Coded UI Training for Microsoft

Best Practices
For Coded UI Tests (continued)

When you create a method, use a meaningful method name


instead of the default name
A meaningful name helps identify the purpose of the method
When possible, limit each recorded method to fewer than 10
actions
This modular approach makes it easier to replace a method if the
UI changes
Create each assertion using the Coded UI Test Builder, which
automatically adds an assertion method to the
UIMap.Designer.cs file
GSAtrain

164

Custom Coded UI Training for Microsoft

Best Practices
For Coded UI Tests (continued)

If the user interface (UI) changes, re-record the test methods,


or the assertion methods, or re-record the affected sections of
an existing test method
Create a separate UIMap file for each module in your
application under test. For more information, see
Testing a Large Application with Multiple UI Maps
In the application under test, use meaningful names when
you create the UI controls
This gives more meaning and usability to the automatically
generated control names

GSAtrain

165

Custom Coded UI Training for Microsoft

Best Practices
For Coded UI Tests (continued)

If you are creating assertions by coding with the API, create


a method for each assertion in the part of the UIMap class
that is in the UIMap.cs file
Call this method from your test method to execute the
assertion

If you are directly coding with the API, use the properties
and methods in the classes generated in the
UIMap.Designer.cs file in your code as much as you can
These classes will make your work easier, more reliable, and
will help you be more productive

GSAtrain

166

Custom Coded UI Training for Microsoft

What to Do If Your
User Interface Changes
User interfaces frequently change during
development
Here are some ways to reduce the effect of these
changes:
Find the recorded method which references this control
and use the Coded UI Test Builder to re-record the
actions for this method
You can use the same name for the method to overwrite
the existing actions
GSAtrain

167

Custom Coded UI Training for Microsoft

What to Do If Your
User Interface Changes (continued)

If a control has an assertion that is no longer


valid:
Delete the method that contains the assertion
Remove the call to this method from the test
method
Add a new assertion by dragging the cross-hair
button onto the UI control, open the UI map, and
add the new assertion
GSAtrain

168

Custom Coded UI Training for Microsoft

What To Do If A Background
Process Needs to Complete Before
The Test Can Continue
// Set the playback to wait for all threads to finish
Playback.PlaybackSettings.WaitForReadyLevel =
WaitForReadyLevel.AllThreads;
// Press the submit button
this.UIMap.ClickSubmit();
// Reset the playback to wait only for the UI thread to finish
Playback.PlaybackSettings.WaitForReadyLevel =
WaitForReadyLevel.UIThreadOnly;
GSAtrain

169

Custom Coded UI Training for Microsoft

Using the Coded UI Test Logs

http://blogs.msdn.com/b/visualstudioalm/a
rchive/2012/06/05/enabling-coded-ui-testplayback-logs-in-visual-studio-2012-relea
se-candidate.aspx

GSAtrain

170

Custom Coded UI Training for Microsoft

Module 17
Evolution of Testing

GSAtrain

171

Custom Coded UI Training for Microsoft

Evolution of Testing

In-Line Programs
Sub-Routines
Functions
Time Sharing
On-Line Terminals
Client Sever
3 Tier Architecture
Thin Client
MVC
SOA
GSAtrain

172

Custom Coded UI Training for Microsoft

De Coupling of
Software Testing
Developers View Individual Units of
Code
Administrators Build Environments and
Versions
Testers Test Complete Applications

GSAtrain

173

Custom Coded UI Training for Microsoft

Primary Drivers of SOA

Inability of TESTERS to test BlackBox Code


o 10,000 lines of code could result in 1 Million Routes
o Impossible to Test

Interface Contract testing is practical


CLOUD increases the environment
exponentially
o Interfaces are constant
GSAtrain

174

Custom Coded UI Training for Microsoft

Microsofts Commitment
To The Cloud
Microsoft Windows 2012
Windows 8
Office 2013
Office365
SQL 2012
SharePoint 2012
LINQ 2012
o are all Cloud SOA Oriented
GSAtrain

175

Custom Coded UI Training for Microsoft

Azure Services

The platform includes five services:


o Live Services for communication and collaboration services
o SQL Azure (formerly SQL Services) for data-storage services
o AppFabric (formerly .NET Services) for cloud-computing
services
o SharePoint Services for web-content-management and
document-management services
o Dynamics CRM Services for customer-relationshipmanagement services
GSAtrain

176

Custom Coded UI Training for Microsoft

Testing is Moving
To The CLOUD
Microsoft TFS is now Cloud Based

GSAtrain

177

Custom Coded UI Training for Microsoft

GSAtrain

178

Custom Coded UI Training for Microsoft

Benefits of TFS in the Cloud

Collaboration is the Key to TFS


o Developers
o Testers Creating Reproducible Automated Tests
o Administrators Builders of (Virtual Machines and Versions)

Using TFS in the cloud can help us enhance


collaboration
It can also enhance collaboration in an organization
with geographically spread locations
GSAtrain

179

Custom Coded UI Training for Microsoft

Concerns with TFS in the


Cloud
With TFS in the cloud is that we can
neither add our own process template nor
make adjustments to the existing
templates

GSAtrain

180

Custom Coded UI Training for Microsoft

Service Oriented
Architecture
Interface Testing
Viewing Objects from the outside
Leads to Unit Testing
Test-Driven Design

GSAtrain

181

Custom Coded UI Training for Microsoft

IT Has a Problem!

ACCELERATING RATE OF CHANGE

GSAtrain

182

Custom Coded UI Training for Microsoft

Reasons for SOA


Rapid Adoption

GSAtrain

183

Custom Coded UI Training for Microsoft

What is the Relationship


Between Cloud and SOA
The short answer is:
Cloud is an initial application of SOA to infrastructure

--its setup, configuration, monitoring , management , with the essential elements of


metering and billing added to satisfy the goal of a charge model of e.g., pay-as-you-go

-- the specific requirements for XaaS: resource pooling and virtualization , elasticity and
multi-tenancy, dynamic configuration and provisioning

Cloud paradigm leverages SOA to deliver a charge-based model for non-business specific
services;
-- infrastructure/hardware

-- software

--platform

https://www.ibm.com/developerworks/mydeveloperworks/blogs/AliArsanjani/?
order=asc&maxresults=15&sortby=0&lang=en
GSAtrain

184

Custom Coded UI Training for Microsoft

SIMM

GSAtrain

https://www2.opengroup.org/ogsys/pu
blications/viewDocument.html?
publicationid=12450&documentid=1
1340

185

Custom Coded UI Training for Microsoft

DOD is Insisting on SOA

Vulnerabilities
Security
Testing

GSAtrain

186

Custom Coded UI Training for Microsoft

Definition of a Service

Discoverable
Self-Describing
Composable
Loose Coupling
Governed by Policy
Independent of Location, Language and Protocol
GSAtrain

187

Custom Coded UI Training for Microsoft

SOA Maturity Model

GSAtrain

188

Custom Coded UI Training for Microsoft

IT Benefits Of SOA

GSAtrain

189

Custom Coded UI Training for Microsoft

SOA

GSAtrain

190

Custom Coded UI Training for Microsoft

SOA

GSAtrain

191

Custom Coded UI Training for Microsoft

SOA

GSAtrain

192

Custom Coded UI Training for Microsoft

GSAtrain

193

Custom Coded UI Training for Microsoft

SOA

GSAtrain

194

Custom Coded UI Training for Microsoft