Professional Documents
Culture Documents
This tutorial gives an overview of the unit testing approach and discusses four testing frameworks
supported by CLion: Google Test, Boost.Test, Catch2, and Doctest.
The Unit Testing in CLion part will guide you through the process of including the frameworks into
your project and describe CLion testing features.
2. Avoid regressions
When you have a suite of unit tests, you can run it iteratively to ensure that everything keeps
working correctly ↗ every time you add new functionality or introduce changes.
A single unit test is a method that checks some specific functionality and has clear pass/fail criteria.
The generalized structure of a single test looks like this:
• Covering all code paths and checking both trivial and edge cases, including those with
incorrect input data (refer to negative testing ↗).
• Assuring that each test works independently and does't prevent other tests from execution.
• Organizing tests in a way that the order in which you run them doesn't affect the results.
It's useful to group test cases when they are logically connected or use the same data. Suites
combine tests with common functionality (for example, when performing different cases for the
same function). Fixture classes help organize shared resources for multiple tests. They are used to
set up and clean up the environment for each test within a group and thus avoid code duplication.
Unit testing is often combined with mocking ↗. Mock objects are lightweight implementations of
test targets, used when the under-test functionality contains complex dependencies and it is
difficult to construct a viable test case using real-world objects.
Frameworks
Manual unit testing involves a lot of routines: writing stub test code, implementing main() , printing
output messages, and so on. Unit testing frameworks not only help automate these operations, but
also let you benefit from the following:
• Various checkers
Checkers are macros for comparing the expected and the actual result. Checkers provided
by testing frameworks often have configurable severity (warning, regular expectation, or a
requirement). Also, they can include tolerances for floating point comparisons and even
pre-implemented exception handlers that check raising of an exception under certain
conditions.
• Tests organization
With frameworks, it's easy to create and run subsets of tests grouped by common
functionality (suites) or shared data (fixtures). Also, modern frameworks automatically
register new tests, so you don't need to do that manually.
• Customizable messages
Frameworks take care of the tests output: they can show verbose descriptive outputs, as
well as user-defined messages or only briefed pass/fail results (the latter is especially handy
for regression testing).
• XML reports
Most of the testing frameworks provide exporting results in XML format. This is useful when
you need to further pass the results to a continuous integration system such as TeamCity ↗
or Jenkins ↗.
• and code generation for tests and fixture classes (available for Google Tests).
As an example, we will use the DateConverter project that you can clone from github repo ↗. This
program calculates the absolute value of a date given in the Gregorian calendar format and converts
it into a Julian calendar date. Initially, the project doesn't include any tests - we will add them step by
step. To see the difference between the frameworks, we will use all four to perform the same tests.
You can find the final version of the project in the DateConverter_withTests ↗ repository. Here is
how the project structure will be transformed:
For each framework, we will do the following:
2. Create two test files, AbsoluteDateTest.cpp and ConverterTests.cpp. These files will be
named similarly for each framework, and they will contain test code written using the syntax
of a particular framework.
Now let's open the cloned DateConverter project and follow the instructions given in the tabs
below:
2. Download Google Test from the official repository ↗. Extract the contents of the
googletest-main folder into Google_tests/lib.
3. Add a CMakeLists.txt file to the Google_tests folder (right-click it in the project tree and
select New | CMakeLists.txt). Add the following lines:
project(Google_tests)
add_subdirectory(lib)
include_directories(${gtest_SOURCE_DIR}/include ${gtest_SOURCE_DIR})
4. In the root CMakeLists.txt script, add the add_subdirectory(Google_tests) line at the end
and reload the project.
2. With two source files added, we can create a test target for them and link it with the
DateConverter_lib library.
3. Copy the Google Test version of our checks from AbsoluteDateTest.cpp ↗ and
ConverterTests.cpp ↗ to your AbsoluteDateTest.cpp and ConverterTests.cpp files.
Now the tests are ready to run. For example, let's click in the left gutter next to the
DateConverterFixture declaration in ConverterTests.cpp and choose Run.... We will get
the following results:
Run/Debug configurations for tests
Test frameworks provide the main() entry for test programs, so it is possible to run them as regular
applications in CLion. However, we recommend using the dedicated run/debug configurations for
Google Test, Boost.Test, Catch2, and Doctest. These configurations include test-related settings
and let you benefit from the built-in test runner (which is unavailable if you run the tests as regular
applications).
CLion atomatically creates Google Test configurations for Cmake targets linked with
gtest or gmock, as well as Doctest configurations for the detected Doctest targets.
• In other fields of the configuration settings, you can set environment variables or command
line options. For example, in the Program arguments field you can set -s for Catch2 tests
to force passing tests to show the full output, or --gtest_repeat to run a Google test
multiple times:
Instead of editing each configuration separately, you can modify the template
itself: settings will be used by default in new configurations based on that template.
When you run a test/suite/fixture using gutter icons, CLion creates temporary Run/Debug
configurations of the corresponding type. You can see these configurations greyed out in the list. To
save a temporary configuration, select it in the Edit Configurations dialog and press :
Test runner
When you run a test configuration, the results (and the process) are shown in the test runner
window that includes:
• tree view of all the running tests with their status and duration,
• toolbar with the options to rerun failed tests, export or open previous results saved
automatically , sort the tests alphabetically to easily find a particular test, or sort them
by duration to understand which test ran longer than others.
You can also execute tests before commit. Set the Run Tests checkbox in the
Commit tool window and select the configuration to run:
When called from a fixture, the menu additionally includes SetUp Method and TearDown Method:
For fixture tests, code generation converts TEST() macros into the appropriate TEST_F() ,
TEST_P() , TYPED_TEST() , or TYPED_TEST_P() (refer to Typed Tests ↗).
Other features
Quick Documentation for test macros
To help you explore the macros provided by testing frameworks, Quick Documentation pop-up (
Ctrl Q ) shows the final macro replacement and formats it properly. It also highlights the strings
and keywords used in the result substitution:
Show Test List
To reduce the time of initial indexing, CLion uses lazy test detection. It means that tests are
excluded from indexing until you open some of the test files or run/debug test configurations. To
check which tests are currently detected for your project, call Show Test List from Help | Find
Action. Note that calling this action doesn't trigger indexing.