Firetower is a system designed to be a point of aggregation for errors from a large or distributed system.
One of the key features of Firetower is its ability to classify incoming errors and normalize their counts in a timeseries as to make monitoring and analysis of service errors much easier.
The goal of the project is to bridge the gap in systems analysis between syslog and custom map-reduce style batch analysis systems in terms of performance, accuracy, and ease of setup. It should be noted that this project is still mostly in the prototype stages of development.
The basic form of the error messages are expected to be somewhat backwards compatible with syslog protocol. See the Syslog RFC.
This is a representation of a basic workflow for Firetower:
I use the latest stable version of Redis for testing:
wget http://redis.googlecode.com/files/redis-2.2.10.tar.gz tar xvf redis-2.2.10.tar.gz cd redis-2.2.10/src make ./redis-server
I recommend doing all development in a virtualenv
git clone [email protected]:gmcquillan/firetower.git virtualenv firetower cd firetower ./bin/activate
If for some reason python-Levenshtein doesn't install, that's ok. We'll just default to using difflib for string comparisons.
Setup: If you're using a virtualenv, activate it.
Make sure that your redis server is running on your local machine on the default port (6379).
Running the Demo:
- Turn on the firetower-client:
cd firetower
. bin/activate
firetower-client -c configs/config.yaml
# This should be spitting out JSON to Stdout like this:
# {'severity': None, 'hostname': 'testmachine', 'syslogtag': 'test', 'sig': 'ToastToast519', 'programname': 'firetower client', 'msg': 'I/O Exception from some file', 'logfacility': 'local1'}
- Turn on the firetower-server:
cd firetower . bin/activate firetower-server -c configs/config.yaml # This will start the classification there will be no output.
- Turn on the web frontend:
cd firetower . bin/activate firetower-web
- Browse to http://localhost:5000 to see what kind of data is being produced.
- To remove your test data (don't do this on a redis server where you have ANY data you care about):
ipython >>> import redis >>> conn = redis.Redis() >>> conn.flushdb()
We usually just use a test discovery tool called nosetests Make sure that you have nosetests installed:
pip install nose
cd firetower nosetests
Alternatively, you can just run:
python -m unittest discover
- Display/frontend
- Plugable Classifiers
- Installation Scripts