An all-in-one monitoring system made specifically for Capture The Flag competitions. Includes a dashboard, live attack/attempt visualisation, a Wall of Sheep and some minimal CTFd integration.
- Network monitoring and analysis (Bro)
- Real-time dashboard (ELK)
- Network requests (Security Onion logstash filters)
- VM-uptime (CTFd API + heartbeat)
- Host system resources (metricbeat)
- Visualisation of attack/challenge attempts (Ergo Witness)
- Wall of Sheep (DoFler)
- Install Docker + Docker Compose
- Clone this repo, including submodules:
git clone --recursive https://github.com/HertogeSimon/CTF-Monitoring.git
- If necessary, change the adapter at services.bro.command in docker/docker-compose.yml
- Run
docker-compose up
in docker/ (this can take a while...) - Navigate to http://localhost:5601/
- Go to Management -> Saved Objects, and import export.json
- Go to Dashboard, click on 'logstash*' and make it the default index (star icon), and refresh
- Download and start the ergo binary for your system
- Fill in the right IP at port 9200
- Click on DONE and ⯈
- If necessary, change the adapter at Monitoring.interface in default.json (DoFler)
- Navigate to http://localhost:3000/
- Make sure you have installed
jq
- Add your challenges to CTFd; make sure to create links of the challenge VM URLs in the markdown description
- Edit your username, password and url in ctf2hb.sh
- Run ctf2hb.sh
Kibana: http://localhost:5601/
DoFler: http://localhost:3000/
CTFd: http://localhost:8000/
Elasticsearch API: http://localhost:9200/
All of these can be changed in docker-compose.yml for outside connections.
Recreate all containers (with old images)
docker-compose up --force-recreate --remove-orphans --renew-anon-volumes
Rebuild a single image:
docker build --no-cache <container_folder>
Follow logs of a single (running) container:
docker logs <container_name> -f
Connect to a running container:
docker exec -it <container_name> bash
=> This only works when the image has no defined entrypoint.
Since Ergo Witness also relies on data from Packetbeat, the visuals are probably not as they should be.
Another problem you might encounter is the difference between local and UTC time in which Ergo Witness and Elasticsearch operate. This can create problems for Ergo's incoming data at midnight.
If you want to fix these issues, you'll need Unity v5.6.0 to recompile my fork of the project. For a more in-depth explanation, read this blog post and the official site.
At first start, DoFler can output 'Access Denied' errors. This is because DoFler depends on the MariaDB container, which takes a while to start. Unfortunately this means you need to restart the container, but then the problem should be solved. The only (and official) solution is to delay the DoFler container with a hacky script, since docker's 'depends_on' relationship does not really do much.
How much longer do I have to wait for this to start?
The first installation can easily take around 15 minutes; subsequent starts should only take a minute at max.
Can I also analyze prerecorded pcap files with this?
Yes! Change the startup command of Bro from "-i <network adapter>" to "-r my.pcap", as currently commented out in docker-compose.yml.
Some useful pcap files can be found on the following sites:
https://www.netresec.com/?page=PcapFiles
https://wiki.wireshark.org/SampleCaptures
http://tcpreplay.appneta.com/wiki/captures.html
How can I add Bro scripts?
- Add a volume to the Bro container overwriting local.bro
- Add a filter to the logstash pipeline if necessary
- Add the new fields to logstash-template.json
How can I change the dashboards?
- Add a visualization
- Add it to a dashboard
Note: At the moment, a lot of bro output does not get processed. You can enable these by moving logstash filters from the disabled to the default pipeline directory.
How do I change X?
Most settings are configured inside docker-compose.yml, and when they're not, they're usually mounted separately as a (docker) volume, so they can be easily modified.
Can I add the indexes and dashboards automatically to Kibana?
Not easily, but it can be done. Either create your own script that talks to the Kibana API, or use metricbeat's import functionality. The latter option is recommended, but you'll have to delay metricbeat's startup since it NEEDS a running Kibana to work.
I heard the ELK stack is written in Java. How do I add more RAM?
Good thinking! You can increase the 'Xmx' and 'Xms' parameters in docker-compose.yml.