%
% How to write a testsuite for ns.
% Drafted by Xuan Chen (xuanc@usc.edu) 
% Fri Nov 10 11:14:57 PST 2000
%
\chapter{Test Suite Support}
\label{chap:testsuite}

The ns distribution contains many test suites under \nsf{tcl/test}, which
used by validation programs (\nsf{validate, validate-wired, validate-wireless, 
and validate.win32}) to verify that the installation of ns is correct. If you 
modify or add new modules to ns, you are encouraged to run the validation 
programs to make sure that your changes do not affect other parts in ns. 

\section{Test Suite Components}
\label{sec:testsuitecomponents}

Each test suite under \nsf{tcl/test} is written to verify the correctness
of a certain module in ns. It has 3 components: 

\begin{itemize}
\item A shell script (test-all-xxx) to start the test;
\item A ns tcl script (test-suite-xxx.tcl) to actually run through the tests 
   defined.
\item A subdirectory (test-output-xxx) under \nsf{tcl/test}, which contains 
 thecorrect trace files generated by the test suite. These files are used to 
 verify if the test suite runs correctly with your ns.
\end{itemize}

(Note: xxx stands for the name of the test suite.)

\section{Write a Test Suite}
\label{sec:writeatestsuite}

You can take one of the test suites under \nsf{tcl/test} as a template when
you are writing your own, for example the test suite written for wireless lan
(test-all-wireless-lan, test-suite-wireless-lan.tcl, and 
test-output-wireless-lan). 

To write a test suite, you first need to write the shell script (test-all-xxx).
In the shell script, you specify the module to be tested, the name of the ns 
tcl script and the output subdirectory. You can run this shell script in quiet 
mode. Below is the example (test-all-wireless-lan):

\begin{program}
   \# To run in quiet mode:  "./test-all-wireless-lan quiet".

   f="wireless-lan"		\# Specify the name of the module to test.
   file="test-suite-\$f.tcl"	\# The name of the ns script.
   directory="test-output-\$f" 	\# Subdirectory to hold the test results
   version="v2"			\# Speficy the ns version.
   
   \# Pass the arguments to test-all-template1, which will run through
   \# all the test cases defined in test-suite-wireless-lan.tcl.
   ./test-all-template1 \$file \$directory \$version \$@
\end{program}


You also need to create several test cases in the ns script (test-suite-xxx.tcl)
by defining a subclass of TestSuite for each different test. For example, in 
test-suite-wireless-lan.tcl, each test case uses a different Ad Hoc routing 
protocol. They are defined as:

\begin{program}	
   Class TestSuite

   \# wireless model using destination sequence distance vector
   Class Test/dsdv -superclass TestSuite

   \# wireless model using dynamic source routing
   Class Test/dsr -superclass TestSuite
   ... ...

\end{program}


Each test case is basically a simulation scenario. In the super class 
TestSuite, you can define some functions, like init and finish to do the work 
required by each test case, for example setting up the network topology and ns
trace. The test specific configurations are defined within the corresponding 
sub-class. Each sub-class also has a run function to start the simulation.

\begin{program}	
   TestSuite instproc init \{\} \{
     global opt tracefd topo chan prop 
     global node_ god_ 
     \$self instvar ns_ testName_
     set ns_         [new Simulator]
      ... ...
   \} 

   TestSuite instproc finish \{\} \{
     \$self instvar ns_
     global quiet

     \$ns_ flush-trace

     puts "finishing.."
     exit 0
   \}
        
   Test/dsdv instproc init \{\} \{
     global opt node_ god_
     \$self instvar ns_ testName_
     set testName_       dsdv
     ... ...    

     \$self next
     ... ...

     \$ns_ at $opt(stop).1 "$self finish"
   \}

   Test/dsdv instproc run \{\} \{
     \$self instvar ns_
     puts "Starting Simulation..."
     \$ns_ run
   \}
\end{program}

All the tests are started by the function runtest in the ns script.

\begin{program}
   proc runtest \{arg\} \{
     global quiet
     set quiet 0

     set b [llength \$arg]
     if \{\$b == 1\} \{
        set test \$arg
     \} elseif \{\$b == 2\} \{
        set test [lindex \$arg 0]
        if \{[lindex \$arg 1] == "QUIET"\} \{
         set quiet 1
        \}
      \} else \{
         usage
     \}
     set t [new Test/\$test]
     \$t run
\}

global argv arg0
runtest \$argv
\end{program}


When you run the tests, trace files are generated and saved to the output 
subdirectory. These trace files are compared to the those correct trace coming 
with the test suite. If the comparation shows difference, the test is failed. 





