backupbot is a easy to use tool for creating (automatic) backups on FreeBSD. Simplicity is one of the most important goals of the project and the following features are supported:
- Manual backups of files and/or MySQL databases.
- Automatic daily backups of files and/or MySQL databases.
- Automatic weekly backups of files and/or MySQL databases.
- Optional compression: gzip, bzip2, xz, lz4, zstd, lrzip, lzip, lzop, pigz, pbzip2 and plzip.
- Optional symmetrical encryption of backups.
- Optional file splitting of (large) file backups.
- Optional retention (auto delete based on file age) for daily and weekly backups.
- Customizable backup location for both file and MySQL database backups.
- Customizable backup ownership for both file and MySQL database backups.
- Customizable file name prefix.
When in doubt, you can use backupbot --help to help you on your way:
root@freebsd:/ # backupbot --help
Usage:
backupbot [feature/option]...
Features:
-b, --backup Backup everything configured in configuration file
-f, --files Backup files configured in configuration file
-m, --mysql Backup MySQL databases configured in configuration file
Options:
--cron Effectuate cron changes from serverbot config
--help Display this help and exit
--version Display version information and exit
With backupbot --version you can check which version of backupbot you're running:
root@freebsd:/ # backupbot --version
backupbot version 1.4.0
Copyright (C) 2019-2022 Nozel.
License CC Attribution-NonCommercial-ShareAlike 4.0 Int.
Written by Sebas Veeke
How to use the other features such as --backup, --files and --mysql is explained in chapter 3.
backupbot consists of two files: /usr/bin/backupbot and /usr/local/etc/backuptbot.conf. These files can be downloaded from the project's repository on GitHub. That being said, never trust some random stranger on the internet and always be cautious when installing new software from a random git repo. In this case backupbot is open source, so you can verify for yourself whether this software should be trusted or not.
The program can be installed by running wget https://raw.githubusercontent.com/nozel-org/freebsd-backupbot/master/backupbot -O /usr/bin/backupbot. Owner and group should be set to root and wheel while read/execute permissions 555 should suffice. This can be done with chown root:wheel /usr/bin/backupbot and chmod 555 /usr/bin/backupbot.
The configuration file can be installed by running wget https://raw.githubusercontent.com/nozel-org/freebsd-backupbot/master/backupbot.conf -O /usr/local/etc/backupbot.conf. Owner and group should be set to root and wheel while read/write permissions for root should suffice. This can be done with chown root:wheel /usr/local/etc/backupbot.conf and chmod 600 /usr/local/etc/backupbot.conf.
wget https://raw.githubusercontent.com/nozel-org/freebsd-backupbot/master/backupbot -O /usr/bin/backupbot
chown root:wheel /usr/bin/backupbot
chmod 555 /usr/bin/backupbot
wget https://raw.githubusercontent.com/nozel-org/freebsd-backupbot/master/backupbot.conf -O /usr/local/etc/backupbot.conf
chown root:wheel /usr/local/etc/backupbot.conf
chmod 600 /usr/local/etc/backupbot.conf
backupbot needs a configuration file to work and this configuration file must be present in /usr/local/etc/backupbot.conf. The default configuration file is commented for a fast and easy configuration, but below the available options are explained in more detail.
backupbot logs its actions to a log file. This log file can be configured with the BACKUPBOT_LOG='/directory/file.log configuration option. If no log file is set, /var/log/backupbot.log will be used.
Automatic backups can be configured to run either daily or both daily and weekly. By setting AUTOMATIC_BACKUP_ENABLE='YES', automatic daily backups will run daily at 00:00. To change the daily backup time, set AUTOMATIC_BACKUP_DAILY='' to a number between 0 (00:00) and 23 (23:00).
To enable automatic weekly backups, set AUTOMATIC_BACKUP_WEEKLY='' to a number between 1 (monday) and 7 (sunday). Setting it to 0 will disable automatic weekly backups. When enabled, the weekly backup will run at the same time as the configured daily backup time.
When both daily and weekly backups are configured, the daily backup won't run on the chosen weekly backup day. Example: you have configured the daily backups to run at 3:00 and the weekly backups to run on on sundays, then the daily backup will run from monday to saturday and skip sundays.
Run backupbot --cron to effectuate any changes made to the automatic backups configuration. This will create relevant cron jobs in the system.
| Automatic backup | Configuration |
|---|---|
AUTOMATIC_BACKUP_ENABLE='' |
Set to YES to enable automatic backups. Set to NO to disable automatic backups. |
AUTOMATIC_BACKUP_DAILY='' |
Set to a number between 0 (00:00) and 23 (23:00) to configure the daily backup process wil start. |
AUTOMATIC_BACKUP_WEEKLY='' |
Set to a number between 1 (monday) and 7 (sunday) to configure the weekly backup day. Set to 0 to disable automatic weekly backups. |
RANDOM_DELAY='' |
Set to 'YES' to add a random delay between 0 and 3600 seconds (1 hour) before the automated backup process starts. Set to NO to disable the random delay. |
File backups can be activated by setting BACKUP_FILES_ENABLE='YES'.
The files that are part of the backup can be configured by setting folder(s) and/or file(s) in BACKUP_FILES=''. For example '/etc', '/usr/local/etc /usr/local/www /var/log' and '/usr/local/etc/apache24 /etc/rc.conf' are all valid entries.
The file backup destination can be configured by setting a folder in BACKUP_FILES_DESTINATION=''. For example '/home/backup' and '/mnt/storage/backup' are valid entries.
Note that all entries should be seperated from each other with a space and that there should be no trailing / at the end of the folder names. If the list with file and folder locations in BACKUP_FILES is quite long, you can also make a list on multiple lines by using backslash newlines.
When automatic daily backups are configured (see 4.1), the retention of automatic daily backups can be configured by setting BACKUP_FILES_RETENTION_DAILY='' to the amount of days you want daily backups to be kept.
Likewise, when automatic weekly backups are configured (see 4.1), the retention of automatic weekly backups can be configured by setting BACKUP_FILES_RETENTION_WEEKLY='' to the amount of days you want weekly backups to be kept. This way it's for example possible to keep weekly backups for a longer period of time than daily backups.
For both daily and weekly backup retention, setting the retention to 0 will keep backups indefinitely.
Backups can be compressed automatically with gzip, bzip2, xz, lz4, zstd, lrzip, lzip, lzop, pigz, pbzip2 or plzip by setting BACKUP_FILES_COMPRESSION='' to:
1 gzip compression algorithm with gzip
2 bzip2 compression algorithm with bzip2
3 xz compression algorithm with xz
4 lz4 compression algorithm with lz4
5 Zstandard compression algorithm with zstd
6 lzma compression algorithm with lrzip (needs package installed)
7 lzma compression algorithm with lzip (needs package installed)
8 lzo compression algorithm with lzop (needs package installed)
101 gzip compression algorithm with pigz (parallel processing)(needs package installed)
102 bzip2 compression algorithm with pbzip2 (parallel processing)(needs package installed)
107 lzma compression algorithm with plzip (parallel processing)(needs package installed)
Compression can be disabled by setting BACKUP_FILES_COMPRESSION='' to 0.
Especially databases can often benefit greatly from compression, but in most cases it's also worthwhile to compress files. Option 1 gzip is a great starting point because it has a decent balance between speed and compression ratio, while also being part of FreeBSD base. The biggest drawback is gzip being single threaded, which is especially apparent on CPUs with a low clock speed. Other compression algorithms might offer better ratios and (much) higher speed, but often require third-party packages to be installed. Option 101 pigz for example is a very fast parallel gzip implementation, but requires pigz to be installed on the server.
For a comparison limited to gzip, bzip2 and xz on a relatively slow server cpu, checkout this Discussion.
Backups can be encrypted automatically with gpg by setting BACKUP_FILES_ENCRYPTION='' to a secret/passphrase of your choosing. Encryption can be disabled by setting BACKUP_FILE_ENCRYPTION='NO'.
The program will check on the availability of gpg on the system when the encryption feature is enabled. Note that this encryption feature uses basic symmetrical encryption with a secret, so make sure other people can't extract the secret from /usr/local/etc/backupbot.conf. You can make /usr/local/etc/backupbot.conf readable only by root by setting chown root:wheel /usr/local/etc/backupbot.conf and chmod 555 /usr/local/etc/backupbot.conf.
Store your secret in a safe place so you will never lose it by accident. You could save the secret in a password manager or on paper in a safe (make sure it's water and heat resistent).
Decryption can be done with gpg, for example by running gpg --output archive.tar.gz --decrypt archive.tar.gz.gpg. Always test decryption before you set and forget the automatic backups by backupbot.
Sometimes large backups can be unwieldy, for example when you need to move them to another server. Backupbot supports the splitting of backup files by setting the BACKUP_FILES_SPLIT='' configuration parameter to a amount of GB as the size of each individual backup part. So if you want to split a large backup in to files of 50 GB each, you set this configuration parameter to 50. This will create multiple backup-file.extension.partxx files that can later be combined again with cat backup-file.extension.part* > backup-file.extension.
File splitting can be disabled by setting BACKUP_FILE_ENCRYPTION='NO'.
File backup ownership can be configured by setting the BACKUP_FILES_OWNER='owner' en BACKUP_FILES_GROUP='group' configuration options. If no owner or group is set, BACKUP_FILES_OWNER='root' and BACKUP_FILES_GROUP='wheel' will be used.
The first part of the backup file name can be configured by setting BACKUP_FILES_PREFIX=''. The default option is '$(date +%y%m%dT%H%M%S)' which translates to the YYMMDDThhmmss format. In practise this could look like 220115T041145, which would mean that the backup was created on 15 january 2022 at 4 hours 11 minutes and 45 seconds in the morning.
| File backup | Configuration |
|---|---|
BACKUP_FILES_ENABLE='' |
Set to YES to enable file backup. Set to NO to disable file backup. |
BACKUP_FILES='' |
Set to folder(s) and/or file(s) separated by a space. No trailing / at the end of folder and/or file names. |
BACKUP_FILES_DESTINATION='' |
Set to a folder without a trailing / at the end of the folder name. |
BACKUP_FILES_RETENTION_DAILY='' |
Set to 0 to keep backups indefinitely. Set to a number (amount of days) of 1 or higher to automatically remove older backups. |
BACKUP_FILES_RETENTION_WEEKLY='' |
Set to 0 to keep backups indefinitely. Set to a number (amount of days) of 1 or higher to automatically remove older backups. |
BACKUP_FILES_COMPRESSION='' |
Set to 1 (gzip), 2 (bzip2) or 3 (xz) to enable compression. Set to 0 to disable compression. See 3.3.4 for more options. |
BACKUP_FILES_ENCRYPTION='' |
Set to secret/passphrase to enable encryption. Set to NO to disable encryption. |
BACKUP_FILES_SPLIT='' |
Set to number in GB to enable file backup splitting. |
BACKUP_FILES_PREFIX='' |
"$(date +%y%m%dT%H%M%S)" |
Mysql database backups can be activated by setting BACKUP_MYSQL_ENABLE='YES'.
For this backup feature to work, MySQL needs to be able to authenticate unattended. You can make this possible by creating a *.cnf file in /usr/local/etc/mysql with the following content:
[client]
user='root'
password='password'
The file backup destination can be configured by setting a folder in BACKUP_FILES_DESTINATION=''. For example '/home/backup' and '/mnt/storage/backup' are valid entries. Note that there should be no trailing / at the end of the folder name.
When automatic daily backups are configured (see 4.1), the retention of automatic daily backups can be configured by setting BACKUP_MYSQL_RETENTION_DAILY='' to the amount of days you want daily backups to be kept.
Likewise, when automatic weekly backups are configured (see 4.1), the retention of automatic weekly backups can be configured by setting BACKUP_MYSQL_RETENTION_WEEKLY='' to the amount of days you want weekly backups to be kept. This way it's for example possible to keep weekly backups for a longer period of time than daily backups.
For both daily and weekly backup retention, setting the retention to 0 will keep backups indefinitely.
Backups can be compressed automatically with gzip, bzip2, xz, lz4, zstd, lrzip, lzip, lzop, pigz, pbzip2 or plzip by setting BACKUP_MYSQL_COMPRESSION='' to:
1 gzip compression algorithm with gzip
2 bzip2 compression algorithm with bzip2
3 xz compression algorithm with xz
4 lz4 compression algorithm with lz4
5 Zstandard compression algorithm with zstd
6 lzma compression algorithm with lrzip (needs package installed)
7 lzma compression algorithm with lzip (needs package installed)
8 lzo compression algorithm with lzop (needs package installed)
101 gzip compression algorithm with pigz (parallel processing)(needs package installed)
102 bzip2 compression algorithm with pbzip2 (parallel processing)(needs package installed)
107 lzma compression algorithm with plzip (parallel processing)(needs package installed)
Compression can be disabled by setting BACKUP_MYSQL_COMPRESSION='' to 0.
Especially databases can often benefit greatly from compression, but in most cases it's also worthwhile to compress files. Option 1 gzip is a great starting point because it has a decent balance between speed and compression ratio, while also being part of FreeBSD base. The biggest drawback is gzip being single threaded, which is especially apparent on CPUs with a low clock speed. Other compression algorithms might offer better ratios and (much) higher speed, but often require third-party packages to be installed. Option 101 pigz for example is a very fast parallel gzip implementation, but requires pigz to be installed on the server.
For a comparison limited to gzip, bzip2 and xz on a relatively slow server cpu, checkout this Discussion.
Backups can be encrypted automatically with gpg by setting BACKUP_MYSQL_ENCRYPTION='' to a secret/passphrase of your choosing. Encryption can be disabled by setting BACKUP_MYSQL_ENCRYPTION='NO'.
The program will check on the availability of gpg on the system when the encryption feature is enabled. Note that this encryption feature uses basic symmetrical encryption with a secret, so make sure other people can't extract the secret from /usr/local/etc/backupbot.conf. You can make /usr/local/etc/backupbot.conf readable only by root by setting chown root:wheel /usr/local/etc/backupbot.conf and chmod 555 /usr/local/etc/backupbot.conf.
Store your secret in a safe place so you will never lose it by accident. You could save the secret in a password manager or on paper in a safe (make sure it's water and heat resistent).
Decryption can be done with gpg, for example by running gpg --output backup.sql.gz --decrypt backup.sql.gz.gpg. Always test decryption before you set and forget the automatic backups by backupbot.
MySQL backup ownership can be configured by setting the BACKUP_MYSQL_OWNER='owner' en BACKUP_MYSQL_GROUP='group' configuration options. If no owner or group is set, BACKUP_MYSQL_OWNER='root' and BACKUP_MYSQL_GROUP='wheel' will be used.
The first part of the backup file name can be configured by setting BACKUP_MYSQL_PREFIX=''. The default option is '$(date +%y%m%dT%H%M%S)' which translates to the YYMMDDThhmmss format. In practise this could look like 220115T041145, which would mean that the backup was created on 15 january 2022 at 4 hours 11 minutes and 45 seconds in the morning.
| File backup | Configuration |
|---|---|
BACKUP_MYSQL_ENABLE='' |
Set to YES to enable mysql backup. Set to NO to disable mysql backup. |
BACKUP_MYSQL_DESTINATION='' |
Set to a folder without a trailing / at the end of the folder name. |
BACKUP_MYSQL_RETENTION_DAILY='' |
Set to 0 to keep backups indefinitely. Set to a number (amount of days) of 1 or higher to automatically remove older backups. |
BACKUP_MYSQL_RETENTION_WEEKLY='' |
Set to 0 to keep backups indefinitely. Set to a number (amount of days) of 1 or higher to automatically remove older backups. |
BACKUP_MYSQL_COMPRESSION='' |
Set to 1 (gzip), 2 (bzip2) or 3 (xz) to enable compression. Set to 0 to disable compression. See 3.4.4 for more options. |
BACKUP_MYSQL_ENCRYPTION='' |
Set to secret/passphrase to enable encryption. Set to NO to disable encryption. |
BACKUP_MYSQL_PREFIX='' |
"$(date +%y%m%dT%H%M%S)" |
The method for recovering your data inside backupbot's backup files depends on used compression and encryption options. Below all combinations will be explained in both the full and short commands.
| Type | Decompress |
|---|---|
| default | tar --extract --verbose --file backup.tar --directory /path/to/destination tar -xvf files.tar -C /path/to/destination |
| gzip | tar --extract --gzip --verbose --file backup.tar.gz --directory /path/to/destination tar -xzvf files.tar.gz -C /path/to/destination |
| bzip2 | tar --extract --bzip2 --verbose --file backup.tar.bzip2 --directory /path/to/destination tar -xjvf files.tar.bzip2 -C /path/to/destination |
| xz | tar --extract --xz --verbose --file backup.tar.xz --directory /path/to/destination tar -xJvf files.tar.xz -C /path/to/destination |
| Type | Decrypt |
|---|---|
| default | gpg --output /path/to/destination/backup.tar --decrypt backup.tar.gpg gpg -o /path/to/destination/backup.tar -d backup.tar.gpg |
| gzip | gpg --output /path/to/destination/backup.tar.gz --decrypt backup.tar.gz.gpg gpg -o /path/to/destination/backup.tar.gz -d backup.tar.gz.gpg |
| bzip2 | gpg --output /path/to/destination/backup.tar.bzip2 --decrypt backup.tar.bzip2.gpg gpg -o /path/to/destination/backup.tar.bzip2 -d backup.tar.bzip2.gpg |
| xz | gpg --output /path/to/destination/backup.tar.xz --decrypt backup.tar.xz.gpg gpg -o /path/to/destination/backup.tar.xz -d backup.tar.xz.gpg |
Note that these commands decompress the existing sql file, without creating a copy.
| Type | Decompress |
|---|---|
| gzip | gzip --decompress --verbose backup.sql.gz gzip -dv backup.sql.gz |
| bzip2 | bzip2 --decompress --verbose backup.sql.bzip2 bzip2 -dv backup.sql.bzip2 |
| xz | xz --decompress --verbose backup.sql.xz gzip -dv backup.sql.gz |
| Type | Decrypt |
|---|---|
| default | gpg --output /path/to/destination/backup.sql --decrypt backup.sql.gpg gpg -o /path/to/destination/backup.sql -d backup.sql.gpg |
| gzip | gpg --output /path/to/destination/backup.sql.gz --decrypt backup.sql.gz.gpg gpg -o /path/to/destination/backup.sql.gz -d backup.sql.gz.gpg |
| bzip2 | gpg --output /path/to/destination/backup.sql.bzip2 --decrypt backup.sql.bzip2.gpg gpg -o /path/to/destination/backup.sql.bzip2 -d backup.sql.bzip2.gpg |
| xz | gpg --output /path/to/destination/backup.sql.xz --decrypt backup.sql.xz.gpg gpg -o /path/to/destination/backup.sql.xz -d backup.sql.xz.gpg |
The configurable prefix (see 3.3.7 and 3.4.7) is the only part of the file name that is not static. Assuming the default prefix, file backups have the following naming schemes:
| Type | Default name | Compression | Encryption | Comp + Crypt |
|---|---|---|---|---|
| Files (manual) | YYMMDDThhmmss-files.tar |
YYMMDDThhmmss-files.tar.gzYYMMDDThhmmss-files.tar.bz2YYMMDDThhmmss-files.tar.xz |
YYMMDDThhmmss-files.tar.gpg |
YYMMDDThhmmss-files.tar.gz.gpgYYMMDDThhmmss-files.tar.bz2.gpgYYMMDDThhmmss-files.tar.xz.gpg |
| Files (daily) | YYMMDDThhmmss-daily-files.tar |
YYMMDDThhmmss-daily-files.tar.gzYYMMDDThhmmss-daily-files.tar.bz2YYMMDDThhmmss-daily-files.tar.xz |
YYMMDDThhmmss-daily-files.tar.gpg |
YYMMDDThhmmss-daily-files.tar.gz.gpgYYMMDDThhmmss-daily-files.tar.bz2.gpgYYMMDDThhmmss-daily-files.tar.xz.gpg |
| Files (weekly) | YYMMDDThhmmss-weekly-files.tar |
YYMMDDThhmmss-weekly-files.tar.gzYYMMDDThhmmss-weekly-files.tar.bz2YYMMDDThhmmss-weekly-files.tar.xz |
YYMMDDThhmmss-weekly-files.tar.gpg |
YYMMDDThhmmss-weekly-files.tar.gz.gpgYYMMDDThhmmss-weekly-files.tar.bz2.gpgYYMMDDThhmmss-weekly-files.tar.xz.gpg |
| MySQL (manual) | YYMMDDThhmmss-mysql-db.sql |
YYMMDDThhmmss-mysql-db.sql.gzYYMMDDThhmmss-mysql-db.sql.bz2YYMMDDThhmmss-mysql-db.sql.xz |
YYMMDDThhmmss-mysql-db.sql |
YYMMDDThhmmss-mysql-db.sql.gz.gpgYYMMDDThhmmss-mysql-db.sql.bz2.gpgYYMMDDThhmmss-mysql-db.sql.xz.gpg |
| MySQL (daily) | YYMMDDThhmmss-daily-mysql-db.sql |
YYMMDDThhmmss-daily-mysql-db.sql.gzYYMMDDThhmmss-daily-mysql-db.sql.bz2YYMMDDThhmmss-daily-mysql-db.sql.xz |
YYMMDDThhmmss-daily-mysql-db.sql |
YYMMDDThhmmss-daily-mysql-db.sql.gz.gpgYYMMDDThhmmss-daily-mysql-db.sql.bz2.gpgYYMMDDThhmmss-daily-mysql-db.sql.xz.gpg |
| MySQL (weekly) | YYMMDDThhmmss-weekly-mysql-db.sql |
YYMMDDThhmmss-weekly-mysql-db.sql.gzYYMMDDThhmmss-weekly-mysql-db.sql.bz2YYMMDDThhmmss-weekly-mysql-db.sql.xz |
YYMMDDThhmmss-weekly-mysql-db.sql |
YYMMDDThhmmss-weekly-mysql-db.sql.gz.gpgYYMMDDThhmmss-weekly-mysql-db.sql.bz2.gpgYYMMDDThhmmss-weekly-mysql-db.sql.xz.gpg |