T3DevKit is a helper for developing the necessary (test suite dependant) pieces of software needed to execute an Abstract Test Suite written in TTCN-3 language. It is made of two parts:
It is tool provider independant. So far it has been validated with three different TTCN-3 tools.
TTCN-3 knowledge is required before using T3DevKit. This section contains a rapid overview of the language and the execution environment. Other presentation materials can be found on the official TTCN-3 website: http://www.ttcn-3.org/.
TTCN-3 is a language designed for testing: it provides the user an easy way to describe test cases, in particular for network related tests. The idea is to test a device, referred as ”System Under Test” (SUT), thanks to a logical implementation written in TTCN-3. The TTCN-3 test logic, running on a machine, will send stimuli to detect if the SUT act accordingly to specifications.
To achieve this objective, a basic test environment is formed of three parts:
A simple scenario is presented in Figure 1.2: a Tester, materialised by a TTCN-3 component is testing a System Under Test (SUT). The Tester and the SUT user communication ports to interact with each other.
The Tester determines if the SUT has a correct behaviour by exchanging messages. The verdict of the test is decided according to the messages received from the SUT.
System Under Test (SUT):
In practice, the SUT can be a communication device (e.g a router). In TTCN-3, the SUT is materialised by a special black-box component: the system component.
All the test processes (stimuli & observations) are implemented on the Tester side, by sending and receiving messages through its communication ports.
The Tester is the software part containing test logics, definitions, functions, etc. The Tester is defined as one (or several) TTCN-3 components.
The definition of a TTCN-3 testcase contains informations about the type of the components that will be instantiated for:
The preamble of a testcase will typically include one or several map instructions to link the components with each other through their ports.
Example: a piece of TTCN-3 code testing if the SUT echoes an expected message. The SUT is defined as being the computer’s terminal and its user.
Test Case Scenario: a quizz. The Tester send a question to the SUT which has to answer correctly to succeed the test.
See Figure 1.3
All these operations are defined at an abstract level: the TTCN-3 source code alone is not sufficient for interacting with the “real” world.
The TTCN-3 source code of a test suite is referred as the “Abstract Test Suite” (ATS).
TTCN-3 is plateform independant. In practice most of the TTCN-3 compilers translate TTCN-3 into C++ or Java, and then use the appropriate C++ or Java compiler to produce an executable. This build process is almost transparent.
But, writing and compiling an Abstract Test Suite in TTCN-3 is not sufficient to make it useable for testing a real implementation. In order to make the connection between the ”abstract” and the ”real” world it is necessary to develop at least a CoDec (CD) and a System Adapter (SA). The main purpose of T3DevKit is to simplify this task.
The standard TTCN-3 execution environment is represented in Figure 1.4
This environment contains several functional blocks with different roles. The main ones are:
These modules interact with the Test Executable via the two standard interfaces named TRI (TTCN-3 Runtime Interface) and TCI (TTCN-3 Control Interface).
Example: Testing an OSPF Router The System Under Test is the OSPF router. The Tester running on a separate machine needs to exchange TTCN-3 messages with the “real” router:
For the SA it is necessary to write a piece of code to open and use a socket in order to send the message.
For the CD the TTCN3 structure has to be translated into the corresponding OSPF binary string (in other words into a ”real” OSPF packet), of course by respecting standards defined in appropriate RFC. This binary string will then be sent to the SUT through the socket.
T3DevKit is a tool to help the development of SA and CD, The typical way of integrating T3DevKit into a project is shown in Figure 1.1.
As it can be seen in figure 1.4, TE and SA interact via TRI interface. T3DevKit provides the user with an higher-level API in order to write SA (a part of the job is already done: port management, memory allocation... see chapter 4) .
The CoDec is also built on top of T3DevLib but it does not need to be written manually. Most of its content is generated automatically from the TTCN-3 sources by the CoDec generator T3CDGen. The only parts that have to be written manually are the “codets”: some pieces of C++ code, that are integrated in the final CoDec by T3CDGen. (see chapter 3)
It is worth to note that at no moment the user needs to manipulate generated code. Generated and written code are built together.
The complete HelloWorld example (including the CD and SA based on T3DevKit) can be found in the ‘examples’ directory of T3DevKit’s package.
Developing and especially maintening a codec is an expensive task. The same applies (at a lower degree) to the system and platform adapter. Several operations need to be performed:
Synchronising type definitions Types definitions are written in TTCN-3. However inside the codec (written in C), the developer needs to know accurately how the messages are represented in TTCN-3 so as to instantiate the correct structures through TCI-CD calls. Each TTCN-3 type definitions is mapped to a sequence of TCI-CD calls, basically the same work must be done twice. This is a tedious and error prone task, especially when the type definitions are updated. With our CoDec generator most of the C++ code is automatically generated. The amount of code to be written manually is reduced to the minimum, so it is a lot faster to write and a lot easier to maintain.
Manipulating TTCN-3 data TTCN-3 values are presented in an abstract way in the TCI. This is close to the TTCN-3 language, but not easy to manipulate in C. For example an integer value is retrieved by calling tciGetIntAbs() which returns an ASCII representation of the integer (eg. "42") and tciGetIntSign() which returns the sign. This not a natural way of manipulating an integer in C. Some casting has to be done before, and it should be done carefully (problems with overflows, ...). With our library, integers are represented with standard C integers, strings with C strings (pointer + length)... and some functions are provided to ease casting between types and handling non byte-aligned data (like bitstrings).
Abstraction The test developer and codec developper do not share the same view of data. The first one works with abstract values and does not care about how they are encoded, while the second one manipulates bit encoded data. Most of the time the codec developer does not care whether a given field is abstractly represented with an integer, a charstring or an octetstring. He justs sees a stream of bits that has to be split into fields. However if he works directly with TCI calls, he will have to know the exact type of the field, do the adequate type conversion and TCI call so as to instantiate the value properly. Our library provides an object oriented framework that allows manipulating any TTCN-3 variable in a uniform way.
Multiplexing calls Most of TRI and TCI functions provided to the user are multiplexed. For example two (or more) communication ports share the same primitives: TriMap(), TriUnmap(), TriSend(), TriCall(), ... This add some unnessary complexity that is not wanted by the developper. With our library implementing a port is simply done by writing a class derived from t3devlib::Port which reimplements functions that are needed.
Portability Although the TTCN-3 interfaces are standardised they are some incompatibilities between TTCN-3 compilers from different vendors. They may occur because of bugs, of imprecision in the standard, or simply because some integration issues are not covered by the standard. With our library the user does not worry about the interfaces, they are implemented in a separate module. Some compilers are already supported and porting the library to another compiler should be simple since all the changes to be done are located at the same place.
Debugging & Exceptions Handling Powerful and accurate debugging features are a must for having a short and reliable development cycle. T3DevLib provides debugging functions for tracing TRI/TCI exchanges with the Test Executable. Also CoDec errors are reported with a set of exceptions, which contain a detailed report of the errors and which are easy to process.
There are still some limitations in T3DevKit: