Add more documentation

doc/configuration.rst

@@ -1,14 +1,23 @@
 Configuration
 ================================
 
-The Caliper run time system can be configured through environment
-variables and/or configuration files. In addition, Caliper comes with
-some built-in configuration profiles. However, the default
-configuration does not activate any measurement or output/processing
-services, so unless you run with third-party tool that uses the
-Caliper API you will at least need to select a configuration profile
-to record Caliper data.
+This section describes the general configuration of the Caliper
+runtime system and lists the core system's configuration
+variables. See next section for the list of measurement services and
+their configuration.
+
+**Note**: by default, Caliper does not activate any measurement or
+output/processing services. Unless you run with third-party tool
+that uses the Caliper API to retrieve data you will at least need to
+select a configuration profile to obtain output.
 
+----------------------------------------
+Configuring the Caliper runtime system
+----------------------------------------
+
+The Caliper runtime system can be configured through environment
+variables and/or configuration files. In addition, Caliper comes with
+some built-in configuration profiles.
 
 Configuration variables
 --------------------------------
@@ -24,6 +33,15 @@ the variable. List elements are typically separated by the colon (':')
 character. Boolean values can be given as '1', 'true', or '0', 'false'
 for *true* and *false*, respectively.
 
+Set caliper configuration variables through environment variables,
+e.g.
+
+.. code-block:: sh
+
+                $ export CALI_LOG_VERBOSITY=2
+
+or configuration files (see below).   
+
 
 Configuration files
 --------------------------------
@@ -58,7 +76,7 @@ switch. You can declare a profile in a configuration file with ``#
 declaration then belong to this profile. Entries that do not belong to
 any specific profile are inserted into the default profile. Caliper
 will use the profile given by the `CALI_CONFIG_PROFILE` variable, or
-the default profile if it is not set.
+the default profile if none is set.
 
 Example::
 
@@ -104,3 +122,39 @@ extended. For example, the following configuration file ::
 adds the ``CALI_MPI_WHITELIST`` entry to the built-in `mpi-trace`
 profile.
 
+----------------------------------------
+Configuration variables reference
+----------------------------------------
+
+This section describes the configuration variables of the Caliper
+runtime system.
+
+CALI_CONFIG_PROFILE=<profile name>
+  Configuration profile name. 
+
+CALI_CONFIG_FILE=<list of config files>
+  Colon-separated list of configuration files. The provided
+  configuration files are read in order. Note: this variable can only
+  be set as an environment variable. Default: ``caliper.config``
+
+CALI_LOG_VERBOSITY=(0|1|2)
+  | Verbosity level. Default: 1
+  |   0: no output
+  |   1: basic informational runtime output
+  |   2: debug output
+
+CALI_LOG_LOGFILE=(stdout|stderr|<filename>)
+  Log file name, or 'stdout'/'stderr' for streaming to standard out or
+  standard error, respectively. Default: stderr
+
+CALI_CALIPER_AUTOMERGE=(true|false)
+  Automatically merge attributes into a common context tree.
+  Decreases the size of context records, but may increase the amount
+  of metadata and reduce performance. Default: true
+
+CALI_SERVICES_ENABLE=(service1:service2:...)
+  List of Caliper service modules to enable. Default: not set.
+
+See next chapter for a list of Caliper services and their
+configuration.
+

doc/index.rst

@@ -21,6 +21,7 @@ Contents:
    build
    usage
    configuration
+   services
 
 
 Indices and tables

doc/services.rst

@@ -0,0 +1,239 @@
+Caliper services
+================================
+
+Caliper comes with a number of optional modules (*services*) that
+provide measurement data or processing and data recording
+capabilities. The flexible combination and configuration of these
+services allows you to quickly assemble recording solutions for a wide
+range of usage scenarios.
+
+You can enable the services required for your measurement with the
+``CALI_SERVICES_ENABLE`` configuration variable, e.g.::
+
+  export CALI_SERVICES_ENABLE=event:pthread:recorder
+
+to create event-triggered context traces for a multi-threaded application.
+
+The following sections describe the available service modules and
+their configuration.
+
+Callpath
+--------------------------------
+
+The `callpath` service uses libunwind to add callpaths to Caliper
+context snapshots. By default, the callpath service provides
+call-stack addresses. Set ``CALI_CALLPATH_USE_NAMES=true`` to retrieve
+function names on-line. Call-path addresses are provided in the
+``callpath.address`` attribute, call-path region names in
+``callpath.regname``. Example:
+
+.. code-block:: sh
+
+  $ export CALI_SERVICES_ENABLE=callpath:event:recorder
+  $ export CALI_CALLPATH_USE_NAME=true
+  $ export CALI_CALLPATH_USE_ADDRESS=true
+  $ ./test/cali-basic
+  $ cali-query -e --print-attributes=callpath.address:callpath.regname
+  $ callpath.address=401207/2aaaac052d5d/400fd9,callpath.regname=main/__libc_start_main/_start
+
+The example shows the ``callpath.address`` and ``callpath.regname``
+attributes in Caliper context records.
+
+Configuration
+................................
+
+CALI_CALLPATH_USE_NAME=(true|false)
+  Provide region names for call paths. Incurs higher overhead. Note
+  that region names for C++ and Fortran functions are not demangled.
+  Default: false.
+
+CALI_CALLPATH_USE_ADDRESS=(true|false)
+  Record region addresses for call paths. Default: true.
+
+CALI_CALLPATH_SKIP_FRAMES=<number of frames>
+  Skip a number of stack frames. This avoids recording stack frames
+  within the Caliper library. Default: 10
+
+Event
+--------------------------------
+
+The event trigger service triggers context snapshots when attributes
+are updated. You can select a list of triggering attributes, or have
+any attribute update trigger context snapshots.
+
+Example
+................................
+
+All attributes trigger context snapshots:
+
+.. code-block:: sh
+
+                $ export CALI_SERVICES_ENABLE=event:recorder
+                $ ./test/cali-basic
+                $ cali-query -e 150819-113409_40027_W5Z0mWvoJUyn.cali
+                phase=main
+                phase=init/main
+                phase=main
+                phase=loop/main
+                iteration=0,phase=loop/main
+                iteration=1,phase=loop/main
+                iteration=2,phase=loop/main
+                iteration=3,phase=loop/main
+                phase=loop/main
+                phase=main
+
+Only "iteration" attribute updates trigger context snapshots:
+
+.. code-block:: sh
+                
+                $ export CALI_SERVICES_ENABLE=event:recorder
+                $ export CALI_EVENT_TRIGGER=iteration
+                $ ./test/cali-basic
+                $ cali-query -e 150819-113409_40027_W5Z0mWvoJUyn.cali
+                phase=loop/main
+                iteration=0,phase=loop/main
+                iteration=1,phase=loop/main
+                iteration=2,phase=loop/main
+                iteration=3,phase=loop/main
+
+Configuration
+................................
+
+CALI_EVENT_TRIGGER=(attribute1:attribute2:...)
+  List of attributes that trigger measurement snapshots.
+  If empty, all user attributes trigger snapshots. Default: empty
+
+Debug
+--------------------------------
+
+The debug service prints an event log on the selected Caliper log
+stream. This is useful to debug source-code annotations. Note that you
+need to set Caliper's verbosity level to at least 2 to see the log
+output.
+
+Example:
+
+.. code-block:: sh
+
+                $ export CALI_SERVICES_ENABLE=debug
+                $ export CALI_LOG_VERBOSITY=2
+                $ ./test/cali-basic
+                == CALIPER: Available services: callpath papi debug event pthread recorder timestamp mpi
+                == CALIPER: Registered debug service
+                == CALIPER: Initialized
+                ...
+                == CALIPER: Event: create_attribute (attr = phase)
+                == CALIPER: Event: pre_begin (attr = phase)
+                == CALIPER: Event: pre_begin (attr = phase)
+                == CALIPER: Event: pre_end (attr = phase)
+                == CALIPER: Event: pre_begin (attr = phase)
+                == CALIPER: Event: create_attribute (attr = iteration)
+                == CALIPER: Event: pre_set (attr = iteration)
+                == CALIPER: Event: pre_set (attr = iteration)
+                == CALIPER: Event: pre_set (attr = iteration)
+                == CALIPER: Event: pre_set (attr = iteration)
+                == CALIPER: Event: pre_end (attr = iteration)
+                == CALIPER: Event: pre_end (attr = phase)
+                == CALIPER: Event: pre_end (attr = phase)
+                == CALIPER: Event: finish
+                == CALIPER: Finished
+
+MPI
+--------------------------------
+
+The MPI service records MPI operations and the MPI rank. Use it to
+keep track of the program execution spent in MPI. You can select the
+MPI functions to track by setting ``CALI_MPI_WHITELIST`` or
+``CALI_MPI_BLACKLIST`` filters.
+
+MPI function names are stored in the ``mpi.function`` attribute, and
+the MPI rank in the ``mpi.rank`` attribute.
+
+Note that you have to link the `libcaliper-mpiwrap` library with the
+application in addition to the regular Caliper libraries to obtain MPI
+information.
+
+Configuration
+................................
+
+CALI_MPI_WHITELIST=(MPI_Fn_1:MPI_Fn_2:...)
+  List of MPI functions to instrument. If set, only whitelisted
+  functions will be instrumented.
+
+CALI_MPI_BLACKLIST=(MPI_Fn_1:MPI_Fn_2:...)
+  List of MPI functions that fill be filtered. Note: if both
+  whitelist and blacklist are set, only whitelisted functions will
+  be instrumented, and the blacklist will be applied to the
+  whitelisted functions.
+
+Pthread
+--------------------------------
+
+The `pthread` service manages thread environments for any
+pthread-based multi-threading runtime system. A thread environment
+manager such as the `pthread` service is responsible for creating
+separate per-thread contexts in multithreaded programs.
+
+If you record attributes on multiple threads, it is strongly
+recommended to enable the `pthread` service.
+
+Recorder
+--------------------------------
+
+The recorder service writes context snapshots into a context trace file.
+
+By default, the recorder service stores context records in an
+in-memory buffer to avoid application performance perturbance because
+of I/O. You can configure the buffer sizes and determine whether they
+are allowed to grow. You can also set the directory and filename that
+should be used; by default, the recorder service will auto-generate a
+file name.
+
+Configuration
+................................
+
+CALI_RECORDER_FILENAME=(stdout|stderr|filename)
+  File name for context trace. May be set to ``stdout`` or ``stderr``
+  to print to the standard output or error streams, respectively.
+  Default: not set, auto-generates a unique file name.
+
+CALI_RECORDER_DIRECTORY=(directory name)
+  Directory to write context trace files to. The directory must exist,
+  Caliper does not create it. Default: not set, use current working
+  directory.
+
+CALI_RECORDER_RECORD_BUFFER_SIZE=(number of records)
+  Initial number of records that can be stored in the in-memory record
+  buffer. Default: 8000
+
+CALI_RECORDER_DATA_BUFFER_SIZE=(number of data elements)
+  Initial number of data elements that can be stored in the in-memory record
+  buffer. Default: 60000
+
+CALI_RECORDER_BUFFER_CAN_GROW=(true|false)
+  Allow record and data buffers to grow if necessary. If false, buffer content
+  will be flushed to disk when either buffer is full.
+  Default: true
+
+Timestamp
+--------------------------------
+
+The timestamp service adds a time offset, timestamp, or duration to
+context records. Note that timestamps are *not* synchronized between
+nodes in a distributed-memory program.
+
+Configuration
+................................
+
+CALI_TIMER_DURATION=(true|false)
+  Include duration (in microseconds) of the context epoch in context
+  snapshots. Default: true
+
+CALI_TIMER_OFFSET=(true|false)
+  Include the time offset (time since program start, in microseconds)
+  with each context snapshot. Default: false
+
+CALI_TIMER_TIMESTAMP=(true|false)
+  Include absolute timestamp (time since UNIX epoch, in seconds) with each
+  context snapshot.
+