stap-server initscript
Version 1.7

INDEX
=====
1. Introduction
2. Usage
3. Files
4. Configuration Format
5. Authentication and Authorization
6. Usage Examples

1. Introduction
===============
The stap-server init script aims to provide
- management of systemtap compile servers as a service.
- convenient control over configured servers and individual (ad-hoc) servers.

2. Usage
========
2.1 Synopsis
------------
  /sbin/service stap-server {start|stop|restart|condrestart|try-restart|force-reload|status} [options]

2.2 Actions
-----------
One of the actions below must be specified.

2.2.1 start
 Start server(s). If a specified server is already started, this action will
 be ignored for that server. If any server fails to start, this action fails.
 If no server is specified, the configured servers are started. If no servers
 are configured, a server for the kernel release and architecture of the host
 is started.

2.2.2 stop
 Stop server(s). If a specified server is already stopped, this action
 will be successful for that server. If a server fails to stop, this action
 fails. If no server is specified, all currently running servers are stopped.

2.2.3 restart
 Stop and start servers again. The specified servers are stopped and restarted.
 If no server is specified, all currently running servers are stopped and
 restarted. If no servers are running this action behaves like 'start'.

2.2.4 condrestart
 Stop and start servers again. The specified servers are stopped and restarted.
 If a specified server is not running, it is not started. If no server is
 specified, all currently running servers are stopped and restarted.  If no
 servers are running, none will be started.

2.2.5 try-restart
 This action is identical to condrestart.

2.2.6 force-reload
 Stop all running servers, reload config files and restart the service as if
 'start' was specified.

2.2.7 status
 Print information about running servers. Information about the specified
 server(s) will be printed. If no server is specified, information about all
 running servers will be printed.

2.3 Options
-----------
The following options may be used to provide additional configuration and
to specify servers to be managed.

2.3.1 -c configfile
 This option specifies a configuration file in addition to those described
 in section 4 below. This file will be processed after the default
 configuration file. If the -c option is specified more than once, the last
 configuration file specified will be used.

2.3.2 -a architecture
 This option specifies the target architecture of the server and is
 analogous to the -a option of 'stap'. See stap(1) for more details.
 The default architecture is the architecture of the host.

2.3.3 -r kernel-release
 This option specifies the target kernel release of the server and is
 analogous to the -r option of 'stap'. See stap(1) for more details.
 The default release is that of the currently running kernel.

2.3.4 -u user-name
 Each stap-server instance is normally run by the user name 'stap-server',
 unless otherwise configured (see 3.2 Configuation Files). This option
 specifies the user name used to run the server(s). The user name specified
 must be a member of the group 'stap-server'.

2.3.5 -I path
 This option specifies an additional path to be searched by the server for
 tapsets and is analogous to the -I option of 'stap'. See stap(1) for more
 details.

2.3.6 -R path
 This option specifies the location of the systemtap runtime to be used by the
 server and is analogous to the -R option of 'stap'. See stap(1) for more
 details.

2.3.7 -B options
 This option specifies options to be passed to 'make' when building systemtap
 modules and is analogous to the -B option of 'stap'. See stap(1) for more
 details.

2.3.8 -D macro[=value]
 This options specifies a macro definition to be used by the server when and
 is analagous to the -D option os 'stap'. See stap(1) for more details.

2.3.9 -i
 This option is a shortcut which specifies one server for each kernel
 release installed in /lib/modules/. Previous -I, -R, -B and -u options will be
 applied to each server, however previous -a options are ignored and the default
 architecture is used.

2.3.10 -n nickname
 This option allows the specification of a server configuration by nickname.
 When -n is specified, a currently running server with the given nickname will
 be searched for. If no currently running server with the given nickname is
 found, a server configuration with the given nickname will be searched for in
 /etc/stap-server/conf.d/*.conf. If a server configuration for the given
 nickname is found, the -a, -r, -I, -R, -B and -u options for that server will
 be used as if they were specified on the command line. If no configuration with
 the given nickname is found, and the action is 'start' (or an action behaving
 like 'start' (see below), the server will be started with the given nickname.
 If no configuration with the given nickname is found, and the action is not
 'start' (or an action behaving like 'start'), it is an error. If a nickname is
 not specified for a server, its nickname will be its process id.

2.3.11 -p pid
 This option allows the specification of a server configuration by process id.
 When -p is specified, a currently running server with the given process id will
 be searched for. If no such server is found, it is an error. If a server with
 the given process id is found, the -a, -r, -I, -R, -B and -u options for that
 server will be used as if they were specified on the command line.

2.3.12 --log logfile
 This option allows the specification of a separate log file for each server.
 Each --log option is added to a list which will be applied, in turn, to each
 server specified. If more servers are specified than --log options, the default
 log file (/var/log/stap-server/log) will be used for subsequent servers.

2.3.13 --port port-number
 This option allows the specification of a specific network port for each
 server. Each --port option is added to a list which will be applied, in turn,
 to each server specified. If more servers are specified than
 --port options, a randomly selected port is used for subsequent servers.

2.3.14 --ssl certificate-db-path
 This option allows the specification of a separate NSS certificate database
 for each server. Each --ssl option is added to a list which will be applied,
 in turn, to each server specified. If more servers are specified than --ssl
 options, the default certificate database
 (/var/lib/stap-server/.systemtap/ssl/server) for subsequent servers.

3. Files
========
3.1 stap-server
---------------
/etc/init.d/stap-server

This is the stap-server initscript.

3.2 Configuration files
-----------------------
Configuration files are written in bash script.

3.2.1 Global config file
 /etc/sysconfig/stap-server

 This config file is for global configuration (see section 4.1).

3.2.2 Server config files
 /etc/stap-server/conf.d/*.conf

 The config files under this directory are for each server to be started by
 default (see section see 4.2).

3.3 Message Log
---------------
/var/log/stap-server/log

All messages including server errors and detailed messages are sent
to this file.
Some error and warning messages are also sent to console and syslogd (syslog
output is optional, because this service may start before syslog).

3.4 Status files
----------------
/var/run/stap-server/<server_pid>

4. Configuration Format
=======================
Configuration files allow us to
  - specify global configuration of logging, server configuration files, status
    files and other global parameters.
  - specify which servers are to be started by default.

4.1 Global Configuration file (/etc/sysconfig/stap-server)
----------------------------------------------------------
The global configuration file may contain settings for the following
variables.

4.1.1 CONFIG_PATH
 Specify the absolute path of the directory containing the default server
 configurations.
 (default: /etc/stap-server/conf.d)

4.1.2 STAT_PATH
 Specify the absolute path of the running server status directory.
 (default: /var/run/stap-server)

4.1.3 LOG_FILE
 Specify the absolute path of the log file
 (default: /var/log/stap-server/log)

4.1.4 STAP_USER
 Specify the userid which will be used to run the server(s).
 (default: stap-server)

4.2 Individual server configuration (/etc/stap-server/conf.d/*.conf)
--------------------------------------------------------------------
Each server configuration file configures a server to be started when no
server is specified for the 'start' action. The configuration file is a
bash script fragment. The following variables may be set.

4.2.1 ARCH
 Specify the target architecture for this server. If ARCH is not set, the
 architecture of the host will be used.

4.2.2 RELEASE
 Specify the kernel release for this server. If RELEASE is not set, the release
 of the kernel running on the host will be used.
 
4.2.3 BUILD
 Specify options to be passed to the 'make' process used to build kernel
 modules.
 
4.2.4 INCLUDE
 An array variable which specifies a list of directories to be searched by the server for tapsets.

4.2.5 RUNTIME
 Specify the directory which contains the systemtap runtime code to be used
 by this server.

4.2.6 DEFINE
 An array variable which specifies a list of macros and their definitions to be used by the
 server which building modules.

4.2.7 USER
 Specify the user name to be used to run this server. The specified user must
 be a member of the group 'stap-server'.

4.2.8 NICKNAME
 Specify the nickname to be used to refer to this server.

4.2.9 LOG
 Specify the location of the log file to be used by this server.

4.2.10 PORT
 Specify the network port to be used by this server.

4.2.11 SSL
 Specify the location of the NSS certificate database to be used by this server.

4.3 Configuration Example
-------------------------

4.3.1 Global Config Example (/etc/sysconfig/stap-server)
 ---
 CONFIG_PATH=~<user>/my-stap-server-configs
 LOG_FILE=/tmp/stap-server/log
 ---

4.3.2 Server Config Examples (/etc/stap-server/conf.d/*.conf)
 --- file1.conf
 ARCH=i386
 RELEASE=2.6.18-128.el5
 PORT=5001
 LOG=/path/to/log/file
 ---

 --- file2.conf
 USER=serveruser
 RELEASE=/kernels/2.6.18-92.1.18.el5/build
 INCLUDE=/mytapsets
 INCLUDE+=/yourtapsets
 BUILD='VARIABLE1=VALUE1 VARIABLE2=VALUE2'
 DEFINE=STP_MAXMEMORY=1024
 DEFINE+=DEBUG_TRANS
 RUNTIME=/myruntime
 NICKNAME=my-server
 SSL=/path/to/NSS/certificate/database
 ---

5. Authentication and Authorization
===================================
5.1 Authentication
5.1.1 SSL
  The stap-server service must authenticate itself to each client using SSL
  before it can be trusted.

5.1.2 Unprivileged Users (Module Signing)
  If the stap-server service will be used to service requests for unprivileged
  users (the --privile option) it must also sign compiled modules using
  a trusted signing certificate.

5.1.3 Certification
  The same certificate is used for both of these purposes. It resides in the
  file stap.cert in /var/lib/stap-server/.systemtap/ssl/server, or in the
  directory specified using the --ssl option, or the SSL configuration file
  variable, on the host which is running the stap-server service.

5.2 Authorization
5.2.1 SSL
  When the systemtap-server package is installed, a certificate for the
  default server userid (stap-server) is generated and authorized as a
  trusted SSL peer. In this way the stap-server service is automatically
  trusted as an SSL peer by all clients on the local host.

  In order for the stap-server service to be trusted by clients on remote
  hosts, it's certificate must be authorized on each host. To do this, first
  obtain a list of available servers

     	stap --list-servers

  This will print a list of available, compatible servers on the local network
  and information about them. Among this information will be certinfo="XXXXX".
  On each host from which a client will access the stap-server service,
  run the following command as root

	stap --trust-servers=ssl,all-users --use-server=XXXXX

  where XXXXX is the certinfo of the server you wish to trust.

  This need only be done once on each client host for each stap-server service
  to be accessed.

  A client on each client host can now access the server

	stap --use-server -e 'probe begin { printf ("Hello\n"); exit (); }'

  This should print 'Hello', provided the user has the authority to load the
  resulting module on the client host.

5.1.2 Unprivileged Users (Module Signing)
  If the user on the client host is an ordinary unprivileged user or a member
  of only the group stapusr, or a member of the groups stapusr and stapsys, he
  will get an error when running the example above,
  since he does not have authority to load the unsigned module returned by the
  server. However if he can get the module signed by a trusted signer, he will
  be able to load it.

  The server will sign the module if the --privilege option is specified by
  the client:

	stap --use-server -e 'probe begin { printf ("Hello\n"); exit (); }' --privilege=stapusr

  The server will now check that the script is safe for use by members of
  stapusr (i.e. it doesn't do anything "dangerous") and, if so, will compile it
  and sign it. However we will still get an error saying that the module
  could not be loaded. This time it is because the server is not a trusted
  signer.

  In order to authorize the server as a trusted signer, on each host from which
  a client will access the stap-server service (including localhost), run the
  following command as root

	stap --trust-servers=signer --use-server=XXXXX

  where XXXXX is the certinfo of the server you wish to trust (see 5.1.2).
  This need only be done once on each client host for each stap-server service
  to be trusted as a signer.
  
  The example above should now work as expected.   

5.1.3 Ordinary Users
  Ordinary users are users who are not root or a member of either of the
  groups stapdev or stapusr.

  These users can still use the stap-server service to compile their scripts
  but they can not load the resulting modules, even if the module is signed
  by a trusted signer (server). The following will ask the server to compile
  the script and return the resulting unsigned module.

	stap --use-server -e 'probe begin { printf ("Hello\n"); exit (); }' -p4

  The following will ask the server to check that the script is safe for use by
  members of stapusr (i.e. it doesn't do anything "dangerous") and, if so, to
  compile it and sign it.

	stap --use-server -e 'probe begin { printf ("Hello\n"); exit (); }' -p4 --privilege=stapusr

  In both cases the name of the resulting module will be printed, but it will
  not be loaded.

6. Usage Eamples
================

6.1 Package Installation
------------------------
After installing the systemtap package, install the systemtap-server package.
# yum install systemtap-server
This package will include the initscript and default configuration files.

6.2 Testing
-----------
See if the default service operates correctly.
 # service stap-server start
 # service stap-server status
 # service stap-server restart
 # service stap-server status
 # service stap-server condrestart
 # service stap-server status
 # service stap-server force-reload
 # service stap-server status
 # service stap-server stop
 # service stap-server status
 # service stap-server condrestart
 # service stap-server status       # <---no server should be running

If there are no errors, the service is correctly installed.

6.3 Service Enabling
--------------------
After all test have passed, enable the stap-server initscript.
# chkconfig stap-server on

6.4 Starting Specific Servers
-----------------------------
6.4.1 Starting a server for an installed kernel release

  # service stap-server start -r <release>
   
  where <release> refers to a kernel installed in /lib/modules

6.4.2 Starting servers for all installed kernel releases

  # service stap-server start -i

6.4.3 Starting a server for a kernel release not installed (cross compiling)

  # service stap-server start -a <arch> -r /<builddir>
   
  where <arch> is the target architecture and
        <buildder> is the absolute path to the kernel's build tree.

6.5 Managing Specific Servers
-----------------------------
For all other actions, specifying a server configuration will act on
that server alone (if it is running). For example

  # service stap-server status -r 2.6.18-128.el5
  # service stap-server restart -a i386 -r 2.6.18-92.1.18.el5
  # service stap-server stop -a powerpc -r /kernels/2.6.18-92.1.18.el5/build

6.6 Configuring Default Servers
-------------------------------
6.6.1 Create Server Config Files
 Each file in /etc/stap-server/conf.d/*.conf represents a server to be started
 by default if no servers are specified on the 'start' action. Each such
 config file may set variables which correspond to the command line options.
 For example:

 # vi /etc/stap-server/conf.d/2.6.18-128.el5.conf
 ARCH=     # default arch
 USER=serveruser
 RELEASE=2.6.18-128.el5
 NICKNAME=2.6.18-128.el5

 # vi /etc/stap-server/conf.d/powerpc.conf
 ARCH=powerpc
 USER=     # default user
 RELEASE=  #default release
 NICKNAME=powerpc

 # vi /etc/stap-server/conf.d/native.conf
 ARCH=     #default arch
 USER=     # default user
 RELEASE=  #default release
 NICKNAME=native

6.6.2 Starting Default Servers

 # service stap-server start

6.6.2 Restarting After Changing the Configuration

 To restart the service after global configuration changes and/or when default
 servers have been added, changed or removed:

 # service stap-server force-reload

6.7 Stopping the stap-server Service
------------------------------------

 To stop all running servers:

 # service stap-server stop