May 15, 2012
A repeated problem we run into when using QMake is that it focuses on being a build tool, and because of that does not implement some features for executing parts of the project (like tests). Others ran into this issue as well, as for example this discussion of specifying run configurations from QMake on Qt DevNet indicates. Multi-part projects often consist of dynamic libraries that contain the features, and applications and tests that link this library. Amongst other things, this approach allows tests to link objects which contain the application’s code. Now when the tests are executed, the dynamic linker needs to be able to pick up the freshly built library and link the test with it at execution time. Projects usually run test cases with make test, and ideally the tests should execute out of the box after the sources have been configured and built (out of the box meaning that no tweaking of the environment variables should be needed between make and make test). After all, it should be as easy as possible for developers to execute tests.
This is also a problem in IDEs like Qt Creator, but also others – for every build configuration (having separate shadow build directories), the developer needs to configure the library path before being able to debug the application. This second problem needs to be fixed in the IDEs. For the first one – configuring test runs from the QMake .pro file – we have developed a solution called QMakeTestRunner that takes care of the problem.
QMakeTestRunner assists in setting library paths when running unit tests in products built with QMake. It makes
make test work out of the box right after makefiles have been generated. It is Free Software, published on Github, and small and easy to integrate into projects as a Git submodule. Much of this information will go into the documentation on Github as well.
Motivation, aka “The Problem”
QMake as a build system is simple and complete for building code, but it lacks a simple way to run unit tests dependent on libraries which are part of the project itself. In this case, paths which are created in the build directory need to be added to the dynamic linker library path before the tests can be executed. Especially with shadow builds, this would require manual setup, and make test would not work out of the box after QMake has been run.
QMakeTestRunner, aka “The Solution”
As soon as any target in a QMake project has the testcase attribute assigned to CONFIG, QMake automatically generates a target called check that runs this test. Unfortunately, make check does not allow the manipulation of the linker paths before running the tests either. QMakeTestRunner contains boilerplate code that wraps QMake’s make check target with a new one called make test, and through variables specified in the QMake file allows the specification of paths to be added to the linker path.
QMakeTestRunner is intended to be used as a git submodule to the main project. It should not require any modifications to be used. It requires Python to be in the path, checks for it, and the QMake run will fail if Python cannot be detected. When using other version control systems than Git, QMakeTestRunner should be small enough to simply be copied into the project as a subdirectory.
Project configuration example
The following example adds the directory libs/ in the project build directory to the linker path. It assumes the QMakeTestRunner repo is located under 3rdparty/QMakeTestRunner. The path where the dynamic library is generated in below the output (build) directory, which is why it is prefixed with OUT_PWD:
TEST_LIB_PATHS += $$OUT_PWD/libs include( 3rdparty/QMakeTestRunner/testtarget.pri )
To trigger extra diagnostic output of the test runner, add
TEST_VERBOSE = 1
before including testtarget.pri.
Once testtarget.pri has been included, a test target is defined in the makefiles. Simply run make test to execute the original make check with the necessary paths.
Mac OSX Frameworks
Frameworks on OSX are a different kind of library. The framework path is passed into the dynamic linker using a different environment variable. To specify a framework path, set the TEST_FRAMEWORK_PATHS variable before including testtarget.pri.
TEST_FRAMEWORK_PATHS += $$OUT_PWD/frameworks
Setting up QMakeTestRunner as a Git submodule
Using a Git submodule has the benefit that the code for the runner does not need to be duplicated into the repository of the project that is supposed to use it. It also makes it easy to pull updates to the test runner by simply updating the submodule to a new revision. To add the submodule as 3rdparty/QMakeTestRunner in your project, follow these steps:
> mkdir -p 3rdparty > git submodule add git://github.com/AgileWorkersSoftware/QMakeTestRunner.git 3rdparty/QMakeTestRunner > git submodule init
The directory 3rdparty/QMakeTestRunner should now contain the current revision of the test runner scripts.
License, contributions, issues and support
QMakeTestRunner is Free Software licensed under the GPL, version 3. Contributions to it are welcome, please propose them as Github pull requests. To submit a bug report or feature wish, please use the Github issue tracker for the project.