Top 5 Python Frameworks For Test Automation In

2019

Robot Framework

Used mostly for development that is acceptance test-driven as well as for acceptance
testing, the Robot Framework is one of the top Python test frameworks. Although it is
developed using Python, it can also run on IronPython, which is .Net-based and on Java-
based Jython. Robot as a Python framework is compatible across all platforms –
Windows, MacOS or Linux.

What Are The Prerequisites?

First of all, you will be able to use Robot Framework (RF) only when you have Python
2.7.14 or any version above it installed. Although Python 3.6.4 is frequently used, code
snippets provided in the official blog of RF will make sure that appropriate notes are
added consisting of all the changes required.

You will also need to install pip, the Python package manager.

Finally, a development framework should be must to download. A popular framework

among developers is the PyCharm Community Edition. However, since code snippets are
not IDE-dependent, you can use any IDE which you have worked on earlier.

What Are The Advantages and Disadvantages Of Robot?

Let’s take a look at what are the advantages and disadvantages of Robot as a test
automation framework over other Python frameworks.

Pros

Using a keyword-driven-test approach, it makes the automation process simpler by

helping the testers to easily create readable test cases.

Test data syntax can be used easily.

Consisting of generic tools and test libraries, it has a vast ecosystem where the
individual elements can be used in separate projects.

The framework is highly-extensible since it has many APIs.

Robot helps you to run parallel tests via a Selenium Grid. However, this feature is not
built-in.

Cons
 Robot Framework sounds very convenient but is tricky when it comes to creating
customized HTML reports. However, you could still present xUnit formatted short
reports by using the Robot framework.

Another flaw of Robot framework is the inadequacy of parallel testing.

Is Robot The Top Python Test Framework For You?

If you are a beginner in the automation domain and have less experience in development,
using Robot as a top Python test framework is easier to use than pytest or pyunit, since it
has rich built-in libraries and involves using an easier test-oriented DSL. However, if you
want to develop a complex automation framework, it is better to switch to pytest or any
other framework involving Python code.

If you are new to Robot framework, here is a document that would help you run your first
automation script using Robot framework with Selenium.

Pytest

Used for all kinds of software testing, pytest is another top Python test framework for
test automation. Being open source and easy to learn, the tool can be used by QA teams,
development teams, in individual practice groups and in open source projects. Because
of its useful features like "assert rewriting," most projects on the internet, including big
shots like Dropbox and Mozilla, have switched from unittest (Pyunit) to pytest. Let’s take
a deep dive and find out what’s so special about this Python framework.

What Are The Prerequisites?

Apart from working knowledge in Python, pytest does not need anything complex. All you
need is a working desktop that has a command line interface, python package manager
and an IDE for development.

What Are The Advantages and Disadvantages Of pytest?

Pros

In the Python testing community, before the arrival of pytest, developers included their
tests inside large classes. However, pytest was revolutionary since it made possible to
write test suites in a very compact manner than it was before.

Other testing tools require the developer or tester to use a debugger or check the logs
and detect where a certain value is coming from. Pytest helps you write test cases in
a way that gives you the ability to store all values inside the test cases and finally
inform you which value failed and which value is asserted.

The tests are easier to write and understand since the boilerplate code is not needed
that much.
 Fixtures are functions which you can use by adding an argument to your test function.
Their job is to return values. In pytest, you can make them modular by using one
fixture from another. Using multiple fixtures helps you cover all the parameter
combinations without rewriting test cases.

Developers of pytest released some useful plugins that make the framework
extensible. For example, pytest-xdist can be used to execute parallel testing without
using a different test runner. Unit tests can also be parameterized without duplicating
any code.

Provides developers with certain special routines that make test case writing simpler
and less prone to errors. The code also becomes shorter and easily understandable.

Cons

The fact that special routines are used by pytest means that you have to compromise
with compatibility. You will be able to conveniently write test cases, but you won’t be
able to use those test cases with any other testing framework.

Is Pytest The Top Python Test Framework For You?

Well, you have to start by learning a full-fledged language, but once you get the hang of it,
you will get all the features like static code analysis, support for multiple IDE, and most
importantly, writing effective test cases. For writing functional test cases and developing
a complex framework, it is better than UnitTest but its advantage is somewhat similar to
Robot Framework if your aim is to develop a simple framework.

If you are considering Pytest as a top Python test framework for you then here is a guide
to test automation using pytest and Selenium WebDriver.

UnitTest, or PyUnit

UnitTest, or PyUnit, is the standard test automation framework for unit testing that
comes with Python. It’s highly inspired by JUnit. The assertion methods and all the
cleanup and setup routines are provided by the base class TestCase . The name of each
and every method in the subclass of TestCase  starts with “test.” This allows them to
run as test cases. You can use the load methods and the TestSuite   class to the group
and load the tests. Together, you can use them to build customized test runners. Just like
Selenium testing with JUnit, UnitTest also has the ability to use UnitTest-sml-reporting
and generate XML reports.

What Are The Prerequisites?

There are no such prerequisites since UnitTest comes by default with Python. To use it,
you will need standard knowledge of the Python framework and if you want to install
additional modules, you will need pip installed along with an IDE for development.

What Are The Advantages and Disadvantages Of PyUnit?

Pros

Being part of the standard library of Python, there are several advantages of using
Unittest.

The developers are not required to install any additional modules since it comes with
the box.

UnitTest is xUnit’s derivative and its working principle is similar to other xUnit
frameworks. People who do not have a strong background in Python generally find it
comfortable to work.

You can run individual test cases in a simpler manner. All you need to do is specify the
names on the terminal. The output is concise as well, making the framework flexible
when it comes to executing test cases.

The test reports are generated within milliseconds.

Cons

Usually, snake_case is used for naming python codes. But since this framework is
inspired a lot from Junit, the traditional camelCase naming method persists. This can
be quite confusing.

The intent of the test code sometimes become unclear, since it supports abstraction
too much.

A huge amount of boilerplate code is required.

Is PyUnit The Top Python Test Framework For You?

In my personal opinion, as well as the opinions of other Python developers, pytest

introduced certain idioms which allowed testers to write better automation code in a very
compact manner. Although UnitTest comes as a default test automation framework, the
facts that its working principle and naming conventions are a bit different than standard
Python codes and it requires too many boilerplate codes  make it a not-so-preferred
Python test automation framework.

Behave

We are all aware of behavior-driven development, the latest Agile-based software

development methodology that encourages developers, business participants, and
quality analysts to collaborate among each other. Behave is another one of the top
Python test frameworks that allow the team to execute BDD testing without any
complications. The nature of this framework is quite similar to SpecFlow and Cucumber
for automation testing. Test cases are written in a simple readable language and later
stuck to the code during execution. The behavior is designed by the behavior specs and
the steps are then reused by other test scenarios.
What Are The Prerequisites?

Anyone with basic knowledge in Python should be able to use Behave. Let’s take a look
at the prerequisites.

Before installing behave, you have to install any version of Python above 2.7.14.

Python package manager or pip is required for working with Behave.

A development environment is the last and most important thing that you need. You
can either use Pycharm, which is preferred by most developers or any other IDE of
your choice.

What Are The Advantages and Disadvantages Of Behave?

Like all other behavior is driven test frameworks, the opinion regarding Behave’s
advantage varies from person to person. Let’s take a look at the common pros and cons
of using Behave.

Pros

System behavior is expressed by semi-formal language and a domain vocabulary that

keeps the behavior consistent in the organization.

Dev teams who are working on different modules with similar features are properly
coordinated.

Building blocks are always ready for executing all kinds of test cases.

Reasoning and thinking are featured in details, resulting in better product specs.

Stakeholders or managers have a better clarity regarding the output of QAs and Devs
because of the similar format of the specs.

Cons

The only disadvantage is that it works well only for black box testing.

Is Behave The Top Python Test Framework For You?

Behave works best only for black box testing. Web testing is a great example since use
cases can be described in plain language. However, for integration testing or unit testing,
Behave is not a good choice since the verbosity will only cause complication for complex
test scenarios. Developers, as well as testers, recommend pytest-bdd. It is an alternative
to Behave since it uses all that is good in pytest and implements it for testing a behavior-
driven scenario.

If you are considering using Behave as a top Python test framework, here is a document
for running your first automation script using Behave with Selenium.

Lettuce
Lettuce is another simple and easy to use behavior-driven automation tool based on
Cucumber and Python. The main objective of Lettuce is to focus on the common tasks of
behavior-driven development, making the process simpler and entertaining.

What Are The Prerequisites?

You will need minimum Python 2.7.14 installed along with an IDE. You can use Pycharm
or any other IDE of your choice. Also, for running the tests, you will be required to install
the Python package manager.

What Are The Advantages and Disadvantages?

Pros

Well, just like any other BDD testing framework, Lettuce enables developers to create
more than one scenario and describe the features in the simple natural language.

Dev and QA teams are properly coordinated since the specs are of similar format.

For Black box testing, Lettuce is quite useful for running behavior-driven tests cases.

Cons

There is only one disadvantage of using Lettuce as a Python framework. For the
successful execution of behavior-driven tests, communication is necessary between
dev teams, QA and stakeholders. Absence of this or a communication gap will make
the process ambiguous and questions can be raised from any team.

Is Lettuce The Top Python Test Framework For You?

According to developers as well as automation testers, Cucumber is more useful when it

comes to executing BDD tests. However, if we are talking about Python developers and
QA, there is no better replacement than pytest-bdd. All the great features of pytest like
compactness, easy to understand code are implemented in this framework combined
with the verbosity of behavior driven testing.

Wrapping It Up!

While pytest, Robot framework and UnitTest are meant for functional and unit testing,
while Lettuce and Behave are best for behavior-driven testing only. From the features
stated, we can conclude that for functional testing, pytest is the best. But if you are new
into Python-based automation testing, Robot Framework is a great tool to get started.
Although the features are limited, it will enable you to get ahead on the track easily. For
Python-based BDD testing, Lettuce and Behave both are equally good, but if you already
have experience in pytest, it’s better to use pytest-bdd.