The ait Autotesting Suite

ait, cit, rit and plit are tools to automate the testing of several tunings to
compare what is best for a particular application.

Initial target apps are market data suites that report latency statistics
for several packet rates.

The test environment consists of two machines, one where the server app
runs and another for the client.

The process starts with ait, that runs on the server machine and configures
the machine in a series of "tuning environments".

For every tuning environment ait will use ssh to start a client run. When
finished ait will collect the report generated by the client or server and
feed it to a per app converter to the cit report format.

The cit report format is a CSV file with this information in each line:

  packet_rate,min_lat,avg_lat,max_lat,lat_dev,pkt_count

Then the cit tool runs and processes the cit report file, taking a snapshot
of the tuning environment, that as of this time consists of:

 ( tso, ufo, softirq_net_tx_prio, softirq_net_rx_prio, server_rtprio,
   irqbalance, server_affinity, server_sched, isolcpus, nic_kthread_affinities,
   nic_kthread_rtprios, oprofile, comment, systemtap, maxcpus, vsyscall64,
   futex_performance_hack, idle, lock_stat )

Example:

 ( "lo=0,eth0=1,eth1=1,eth5=1,eth4=1,eth2=0,eth3=0,bond0=0,bond1=0",
   "lo=0,eth0=0,eth1=0,eth5=0,eth4=0,eth2=0,eth3=0,bond0=0,bond1=0",
   "90,90,90,90,90,90,90,90",
   "90,90,90,90,90,90,90,90", 0, 0, ff, SCHED_OTHER, None,
   "eth1=ff;eth5=ff;eth4=ff;eth2=ff;eth3=ff",
   "eth1=95;eth5=95;eth4=95;eth2=95;eth3=95",
   0, , 0, None, 1, 0, poll, False )

This is stored in database tables, so that several test runs can be made on the
same tuning environment and later combined/compared.

The cit file packet rate and latency data is stored in another table, together
with an indication to the report it came from and tuning environment where
it was obtained.

The last tool is the one used to go over the database and plot graphics for
the various data collected, as well as order by best results so that one
can see the tuning environment.