Extending JUNIT to Integration, System, Load and Stress Testing




DI Siegfried GOESCHL

IT Serv GmbH












This paper describes an extension of the JUNIT 3.7 test framework called JUNITPP to facilitate integration, system, stress and load testing. The extension provides a test data repository, command line argument passing and an extended TestRunner with built-in multi-threading support.





1     Introduction

The JAVA unit test framework JUNIT  is extensively used in the JAVA community due to its elegance of design and ease of use. Consequently the usage is extended beyond the original scope to cover integration, system, load and stress testing. Extending the scope naturally results in shortcomings such as the lack of a test data repository which stems from the original design. The absence of a test data repository results in hard-coded test data which increases the maintenance costs of testing database-driven systems. This fact should not be underestimated since the test code should have the same quality requirements as the application code. It is not uncommon to ship the regression test suite as part of the deliverables to increase customer confidence and to supplement the documentation. To overcome this and other limitations the JUNIT test framework was extended to provide a test data repository, command line arguments and an improved TestRunner supporting a built-in repetition counter and multithreading on the command line.

2     Using the JUNITPP Extensions

The main improvement of JUNITPP is the provision of a test data repository accessed by subclassing junit.extensions.ConfigurableTestCase.

public class FooTest extends ConfigurableTestCase {


    public FooTest(String name) {




    public void testFoo() {

        String stringValue = getString("key1");

        Boolean booleanValue = getBoolean("key2");

        Integer integerValue = getInteger("key3");

        Double doubleValue = getDouble("key4");



Figure 1 - Using the test data repository for a simple test

The class ConfigurableTestCase uses java.util.Properties to load the test data. The loading of the property file is triggered in the constructor of ConfigurableTestCase. For a master test suite which only invokes other test suites (usually AllTests) this strategy doesn’t work since no constructor is invoked. In this case loading the properties file has to be done manually by invoking ConfigurationFactory.init(AllTests.class) in the suite() method.


public class AllTests extends ConfigurableTestCase {


    public AllTests(String name) {




    public static Test suite() {


        // load property file



        TestSuite suite = new TestSuite();



        return suite;



Figure 2  Using the test data repository for a master test suite

How is the properties file found when an instance of ConfigurableTestCase is executed? It is assumed that the property file has the same name as the class file but a different extension, i.e. the property file for the FooTest.class would be named FooTest.ini. The property file is either looked up in the current directory or in a list of directories such as “src”, “src/java”, “src/test” and “src/test/java”.

In some projects this approach might not work (e.g. using a single property file for all test suites) therefore the system property ‘junit.conf  either defines a property file or a starting directory for the search.

java –Djunit.conf=MyFooTest.conf junit.swingui.TestRunner FooTest

java –Djunit.conf=./test/data junit.swingui.TestRunner FooTest

Figure 3 – Specifying a test data repository

How is an entry in the property file defined? To access the test data repository the class name, the name of the test case and the property name are concatenated to generate the key in the following order:

·     fully qualified class name + test name + property name

·     class name without package name + test name + property name

·     class name without package name + property name

·     property name

This implementation allows the reuse of test data definitions shared by one or more test suites, e.g. the name of the sever used for testing a client/server application with multiple test suites.

# FooTest.conf






Figure 4 - Content of a test data repository

In the case of a master test suite the corresponding property file contains references to the property files of the contained test suites, which are loaded recursively.


# AllTests.conf




Figure 5 - Content of a test data repository for a master suite

What happens if a different type of test data repository already exists, e.g. in a relational database? In JUNIT++ the implementation of test data repository is determined and instantiated at runtime. Hence it is possible to provide a different implementation of a test data repository by defining the implementation class at the command line:

java -Djunit.conf.class=xyz.JDBCTestProperties junit.extensions.PPTestRunner foo.bar.AllTests

Figure 6 - Using a different implementation of a test data repository

After a successful test execution one might reuse the test suite to gather performance data either by simply measuring the execution time or profiling the application. The profiling gives a good indication of how much time and resources are spent in an ORB runtime library or a Servlet engine. It is necessary to fine-tune the number of test execution to find a balance between the effect of one-time initializations distorting the result and time required to generate the profiling data. By using junit.extensions.PPTestRunner the number of test repetitions can be defined on the command line:

java junit.extensions.PPTestRunner –r 10 foo.bar.AllTests

Figure 7 - Starting PPTestRunner with 10 repetitions

If the profiling of a client/server application is done on one computer it will become overloaded. In this case a delay of the test case invocations can be defined on the command line:

java junit.extensions.PPTestRunner –r 10 -w 100 foo.bar.AllTests

Figure 8 – Executing tests with 10 repetitions and 100 ms invocation delay

The next step might be overloading the server until it crashes due to race conditions and/or resource shortage. This can be accomplished by invoking one or more test suites using multiple threads, repetition counters and test case invocation delays to simulate a more realistic user behaviour.

java junit.extensions.PPTestRunner –r 1000 –s 10 foo.bar.AllTests

Figure 9 – Server crash test with a client executing 1000 times with 10 threads simultaneously

Some components require command line parameters for initialization. JUNITPP provides a uniform way of defining application specific command line parameters by passing them as a system property. Some development environments are unable to process quoted arguments when invoking the debugger. This is why multiple arguments can be separated by the ‘%’ sign.

java –Djunit.params="-c foo.ext" junit.textui.TestRunner FooTest

java –Djunit.params=-c%foo.ext junit.textui.TestRunner FooTest

Figure 10 - Defining command line arguments

Sometimes passing arguments via command line is clumsy, e.g. initialising a JDBC connection for multiple test suites. In this case a ConfigurableTestCase can be used, where the parameters are stored in a corresponding property file.

public class JDBCTestSetup extends ConfigurableTestSetup {


    static private Connection connection = null;


    public JDBCTestSetup() {}


    public void setUp() {

       connection = DriverManager.getConnection(






    public void tearDown() {




    public Connection getConnection() {

        return connection;



Figure 11 - Using a ConfigurableTestSetup

If a test suite does a lot of printing, it is useful to get a more verbose output of a test run. This can be accomplished by turning on the verbose mode of PPTestRunner.

java junit.extensions.PPTestRunner –v junit.test.extensions.ConfigurableTestCaseTest

Figure 12 – Using the verbose mode

3     Conclusion

JUNITPP offers a set of functions necessary to extend the scope of usage from unit testing to integration, system, load and stress testing. These extensions include a test data repository for composite test suites, command line argument support, a ConfigurableTestSetup and an extended TestRunner.

The usage of JUNITPP makes it possible to separate test data from test code, which is essential for testing database-driven applications. Omitting this separation results in hard-coded test data, which increase the test maintenance costs and decrease the acceptance of regression tests.

The ease of generating an arbitrary load under various conditions encourages developers to use stress tests for components at a very early stage of development which in turn increases the reliability of the final product. This observation is particularly relevant for distributed systems because it might require an expert’s knowledge to trace a failure back to a single component.