When building rsync, you'll want to install various libraries in order to get
all the features enabled. The configure script will alert you when the
newest libraries are missing and tell you the appropriate --disable-LIB
option to use if you want to just skip that feature. What follows are various
support libraries that you may want to install to build rsync with the maximum
features (the impatient can skip down to the package summary):
You need to have a C compiler installed and optionally a C++ compiler in order to try to build some hardware-accelerated checksum routines. Rsync also needs a modern awk, which might be provided via gawk or nawk on some OSes.
If you're installing from the git repo (instead of a release tar file) you'll also need the GNU autotools (autoconf & automake) and your choice of 2 python3 markdown libraries: cmarkgfm or commonmark (needed to generate the man pages). If your OS doesn't provide a python3-cmarkgfm or python3-commonmark package, you can run the following to install the commonmark python library for your build user (after installing python3's pip package):
python3 -mpip install --user commonmark
You can test if you've got it fixed by running (from the rsync checkout):
./md2man --test rsync-ssl.1.md
Alternately, you can avoid generating the man pages by fetching the very latest versions (that match the latest git source) from the generated-files dir. One way to do that is to run:
./prepare-source fetchgen
To support copying ACL file information, make sure you have an acl development library installed. It also helps to have the helper programs installed to manipulate ACLs and to run the rsync testsuite.
To support copying xattr file information, make sure you have an attr development library installed. It also helps to have the helper programs installed to manipulate xattrs and to run the rsync testsuite.
The xxHash library provides extremely fast checksum functions that can make the "rsync algorithm" run much more quickly, especially when matching blocks in large files. Installing this development library adds xxhash checksums as the default checksum algorithm. You'll need at least v0.8.0 if you want rsync to include the full range of its checksum algorithms.
The zstd library compression algorithm that uses less CPU than the default zlib algorithm at the same compression level. Note that you need at least version 1.4, so you might need to skip the zstd compression if you can only install a 1.3 release. Installing this development library adds zstd compression as the default compression algorithm.
The lz4 library compression algorithm that uses very little CPU, though it also has the smallest compression ratio of other algorithms. Installing this development library adds lz4 compression as an available compression algorithm.
The openssl crypto library provides some hardware accelerated checksum algorithms for MD4 and MD5. Installing this development library makes rsync use the (potentially) faster checksum routines when computing MD4 & MD5 checksums.
To help you get the libraries installed, here are some package install commands for various OSes. The commands are split up to correspond with the above items, but feel free to combine the package names into a single install, if you like.
-
For Debian and Ubuntu (Debian Buster users may want to briefly(?) enable buster-backports to update zstd from 1.3 to 1.4):
sudo apt install -y gcc g++ gawk autoconf automake python3-cmarkgfm sudo apt install -y acl libacl1-dev sudo apt install -y attr libattr1-dev sudo apt install -y libxxhash-dev sudo apt install -y libzstd-dev sudo apt install -y liblz4-dev sudo apt install -y libssl-dev -
For CentOS (use EPEL for python3-pip):
sudo yum -y install epel-release sudo yum -y install gcc g++ gawk autoconf automake python3-pip sudo yum -y install acl libacl-devel sudo yum -y install attr libattr-devel sudo yum -y install xxhash-devel sudo yum -y install libzstd-devel sudo yum -y install lz4-devel sudo yum -y install openssl-devel python3 -mpip install --user commonmark -
For Fedora 33:
sudo dnf -y install acl libacl-devel sudo dnf -y install attr libattr-devel sudo dnf -y install xxhash-devel sudo dnf -y install libzstd-devel sudo dnf -y install lz4-devel sudo dnf -y install openssl-devel -
For FreeBSD (this assumes that the python3 version is 3.7):
sudo pkg install -y autotools python3 py37-CommonMark sudo pkg install -y xxhash sudo pkg install -y zstd sudo pkg install -y liblz4 -
For macOS:
brew install automake brew install xxhash brew install zstd brew install lz4 brew install openssl -
For Cygwin (with all cygwin programs stopped, run the appropriate setup program from a cmd shell):
setup-x86_64 --quiet-mode -P make,gawk,autoconf,automake,gcc-core,python38,python38-pip setup-x86_64 --quiet-mode -P attr,libattr-devel setup-x86_64 --quiet-mode -P libzstd-devel setup-x86_64 --quiet-mode -P liblz4-devel setup-x86_64 --quiet-mode -P libssl-develSometimes cygwin has commonmark packaged and sometimes it doesn't. Now that its python38 has stabilized, you could install python38-commonmark. Or just avoid the issue by running this from a bash shell as your build user:
python3 -mpip install --user commonmark
After installing the various libraries, you need to configure, build, and install the source:
./configure make sudo make install
The default install path is /usr/local/bin, but you can set the installation directory and other parameters using options to ./configure. To see them, use:
./configure --help
Configure tries to figure out if the local system uses group "nobody" or "nogroup" by looking in the /etc/group file. (This is only used for the default group of an rsync daemon, which attempts to run with "nobody" user and group permissions.) You can change the default user and group for the daemon by editing the NOBODY_USER and NOBODY_GROUP defines in config.h, or just override them in your /etc/rsyncd.conf file.
As of 2.4.7, rsync uses Eric Troan's popt option-parsing library. A cut-down copy of a recent release is included in the rsync distribution, and will be used if there is no popt library on your build host, or if the --with-included-popt option is passed to ./configure.
If you configure using --enable-maintainer-mode, then rsync will try to pop up an xterm on DISPLAY=:0 if it crashes. You might find this useful, but it should be turned off for production builds.
If you want to automatically use a separate "build" directory based on the current git branch name, start with a pristine git checkout and run "mkdir auto-build-save" before you run the first ./configure command. That will cause a fresh build dir to spring into existence along with a special Makefile symlink that allows you to run "make" and "./configure" from the source dir (the "build" dir gets auto switched based on branch). This is helpful when using the branch-from-patch and patch-update scripts to maintain the official rsync patches. If you ever need to build from a "detached head" git position then you'll need to manually chdir into the build dir to run make. I also like to create 2 more symlinks in the source dir: ln -s build/rsync . ; ln -s build/testtmp .
Note that Makefile.in has a rule that uses a wildcard in a prerequisite. If your make has a problem with this rule, you will see an error like this:
Don't know how to make ./*.c
You can change the "proto.h-tstamp" target in Makefile.in to list all the *.c filenames explicitly in order to avoid this issue.
Under packaging you will find .spec files for several distributions. The .spec file in packaging/lsb can be used for Linux systems that adhere to the Linux Standards Base (e.g., RedHat and others).
The HP-UX 10.10 "bundled" C compiler seems not to be able to cope with ANSI C. You may see this error message in config.log if ./configure fails:
(Bundled) cc: "configure", line 2162: error 1705: Function prototypes are an ANSI feature.
Install gcc or HP's "ANSI/C Compiler".
Some versions of Mac OS X (Darwin) seem to have an IPv6 stack, but do not completely implement the "New Sockets" API.
This site says that Apple started to support IPv6 in 10.2 (Jaguar). If your build fails, try again after running configure with --disable-ipv6.
IBM AIX has a largefile problem with mkstemp. See IBM PR-51921. The workaround is to append the following to config.h:
#ifdef _LARGE_FILES #undef HAVE_SECURE_MKSTEMP #endif