Testing and tools

## 9 Testing and tools

### 9.1 Tracing

We have instrumented our code in order to enable tracing. Running the application with tracing deactivated, causes a neglectible performance overhead (an external call to a function which returns an atom). Activation of tracing does not require any recompilation of the code, since we rely on Erlang/OTP's built in support for dynamic trace activation. In our case tracing of calls to a given external function.

Event traces can be viewed in a generic message sequence chart tool, that we have written. It can either be used in batch by loading event traces from file or interactively, as we are doing at demos and when we debug our own code. The event trace stuff can for the moment be found under megaco/utils but, will later be documented and released as an own application.

### 9.2 Measurement and transformation

We have included a simple tool for codec measurement and message transformation.

The tool is located in the example directory..

#### 9.2.1 Requirement

- Erlang/OTP, version R11B or later.
- Version 3.4 or later of this application.
- Version 1.4.4.10 or later of the asn1 application.
- The flex libraries. Without it, the flex powered codecs cannot be used.

#### 9.2.2 Results

The results from the measurement run is four excel-compatible textfiles:

- decode_time.xls -> Decoding result
- encode_time.xls -> Encoding result
- total_time.xls -> Total (Decoding+encoding) result
- message_size.xls -> Message size

#### 9.2.3 Instruction

The tool contain three things:

- The transformation module
- The measurement module
- The basic message file archive

##### 9.2.3.1 Transformation module

The transformation module is used to transform a set of messages encoded with one codec into another other codec's.

Example: Start an erlang node, and make sure it has the path to both the latest megaco ebin-dir as well as the dir containing the transformation module:

            % erl -pa <path-megaco-ebin-dir> -pa <path-to-tranformation-module-dir>
            Erlang (BEAM) emulator version 5.3 [source]

            Eshell V5.3  (abort with ^G)
            1> megaco_codec_transform:t(pretty, [compact, per, ber, erlang]).
            ...
            2> halt().

or to make it even easier if you as above use pretty text as base:

            %  erl -noshell -pa <path-megaco-ebin-dir> -pa <path-to-tranformation-module-dir> \
                   -s megaco_codec_transform tt -s erlang halt

or using ber binary as base:

            %  erl -noshell -pa <path-megaco-ebin-dir> -pa <path-to-tranformation-module-dir> \
                   -s megaco_codec_transform tb -s erlang halt

Now the messages in the 'pretty' directory has been trasnformed and stored into the other codec dir's.

It is possible to transform from any codec to any other.

##### 9.2.3.2 Measurement module

The measurement module is used to measure the decode and encode the messages in the codec dirs.

Example: Start an erlang node, and make sure it has the path to both the latest megaco ebin-dir as well as the dir containing the measurement module:

            % erl -pa <path-megaco-ebin-dir> -pa <path-to-meas-module-dir>
            Erlang (BEAM) emulator version 5.3 [source]

            Eshell V5.3  (abort with ^G)
            1> megaco_codec_meas:t([pretty, compact, per, ber, erlang]).
            ...
            2> halt().

or to make it even easier, assuming a measure shall be done on all the codecs (as above):

            % erl -noshell -pa <path-megaco-ebin-dir> -pa <path-to-meas-module-dir> \
                  -s megaco_codec_meas t -s erlang halt

When run as above (this will take some time), the measurement process is done as followes:

              For each codec:
                  For each message:
                      Read the message from the file
                           Detect message version
                           Measure decode
                           Measure encode
              Write results, encode, decode and total, to file

The measure is done by iterating over the decode/encode function for approx 5 seconds per message and counting the number of decodes/encodes.

##### 9.2.3.3 Message file archive

This is basically a gzipped tar file of a directory tree with the following structure:

              time_test/pretty/<message-files>
                        compact/
                        per/
                        ber/<message-files>
                        erlang/

The only directories containing any files are the pretty-dir and the ber-dir. It's the same messages encoded with different codec's. This means it is possible to choose the message basis for the (transformation and) measurement.

These files include both version 1 and version 2 messages.

It is of course possible to add and remove messages at will. The messages included are the ones used in our own measurements.

#### 9.2.4 Notes

##### 9.2.4.1 Binary codecs

There are two basic ways to use the binary encodings: With package related name and termination id transformation (the 'native' encoding config) or without. This transformation converts package related names and termination id's to a more convenient internal form (equivalent with the decoded text message).

The transformation is done _after_ the actual decode has been done.

Furthermore, it is possible to make use of a linked in driver that performs some of the decode/encode, decode for ber and encode for per (the 'driver' encoding config).

Therefor in the tests, binary codecs are tested with four different encoding configs to determine exacly how the different options effect the performance: with transformation and without driver ([]), without transformation and without driver ([native]), with transformation and with driver ([driver]) and finally without transformation and with driver ([driver,native]).

##### 9.2.4.2 Included test messages

These messages are ripped from the call flow examples in an old version of the RFC.

##### 9.2.4.3 Measurement tool directory name

Be sure not no name the directory containing the measurement binaries starting with 'megaco-', e.g. megaco-meas. This will confuse the erlang application loader (erlang applications are named, e.g. megaco-1.0.2).

Copyright © 1991-2006 Ericsson AB