TestStudio

An Environment for Automatic Test Generation Based on Design by Contract (TM)

Ilinca Ciupa

Diploma thesis

Supervised by Prof. Dr. Bertrand Meyer and Dr. Karine Arnout, ETH Zurich

March 1, 2004 – July 31, 2004

Overview

The importance of testing in the software development process is recognized as a matter of fact nowadays. However, practice shows that, in spite of on-going research in the field, testing methodologies and tools have not yet managed to provide software developers with adequate support for testing activities.

In this project, we suggest a different approach to testing: the testing process is fully automatic. Our methodology is based on Design by Contract (TM). Contracts are a valuable source of information regarding the intended semantics of the software. The information that contracts (preconditions, postconditions, class invariants, loop variants and invariants, and check instructions) provide can be used to check whether the software fulfills its intended purpose. By checking that the software respects its contracts, we can ascertain its validity. Therefore, contracts provide the basis for automation of the testing process.

The goal of this project was to develop an environment for automatic generation of tests, which we call TestStudio. It allows the user to generate, compile and run tests on the push of a button. It also allows fine tuning of the testing process and offers all the benefits of a friendly user interface.

TestStudio is a proof of concept for push-button testing. It shows that, in the presence of contracts, a fully automatic testing process is possible. In fact, it is not only possible, but also effective and efficient. By using TestStudio, we discovered several bugs in Eiffel libraries, bugs that had escaped thorough manual testing performed by library developers.

System requirements

Currently, the only operating system that TestStudio supports is Windows. We have only tested it on Windows XP, but expect that it should also run on Windows 2000. Future versions will be available for other platforms too.

Version 5.4 (or later) of EiffelStudio is required to run TestStudio. This version is needed for assertion checking to be enabled and disabled correctly.

Installation

Download the file “TestStudio.zip”. It contains:
- the source code of TestStudio (for both the GUI and command line versions) in the “src” directory;
- executables for the GUI and command line versions in the “bin” directory;
- the user manual in the “doc” directory;
- the Gobo Eiffel library under the “library” directory;
- file “License.txt” containing the license for the software;
- file “Readme.txt” containing information about TestStudio, its installation and the contents of the delivery;
- file “Release_notes.txt” containing information about the requirements for running TestStudio.
Unzip its content to a directory of your choice and set the $TEST_STUDIO environment variable to point to that directory.
Unzip the package “gobo33.zip” provided with this release and set the $GOBO environment variable so that it points to the directory where you unzipped the file.