rdup.1.in

'\" t
.TH RDUP 1 "24 Dec 2005" "@PACKAGE_VERSION@" "@PACKAGE_NAME@"
.SH NAME
@PACKAGE_NAME@ \- generate a file list suitable for making backups
.SH SYNOPSIS
.B @PACKAGE_NAME@
[\fI\-N timestamp\fR]
-[\fI\-P CMD\fR]...
[\fIOPTION\fR]...
\fIFILELIST\fR
[\fIDIR/FILE]...\fR

.SH DESCRIPTION
\fBrdup\fR is a utility inspired by rsync and the Plan9 way of doing
backups. \fBrdup\fR itself does not backup anything. It only prints
a list of files that are changed, or all files in case of a null dump.
It also handles files that are removed, allowing for correct incremental 
backups. All paths printed are absolute. \fBrdup\fR uses the change time
(ctime) to decide whether a file is altered.
.PP
It works as follows, for a full dump
.TP
.B 1.
Crawl all directories, and print all the names found to standard output.
Also check for \fI._rdup_.\fR files to restore the original ownership
(see the \fI\-u\fR flag).
.TP
.B 2.
Write a filelist with all the names found when crawling.
Use this list to calculate the correct incremental dump.

.PP
And for incremental dumps

.TP
.B 1.
Read in the filelist that was written when doing a full dump.
.TP
.B 2.
Touch the time stamp file.
.TP
.B 3.
Crawl all the directories again.
.TP
.B 4.
Diff 1. and 2. to get two lists; one of removed items and one
of added/modified items.
.TP
.B 5.
Write the removed items to standard output
.TP
.B 6.
Write the modified/new items to standard output.
.TP
.B 7.
Write a new filelist.

.PP
The
.IR FILELIST
is a internal list \fBrdup\fR writes to, to keep track of which files
are in a backup. If you don't want this (i.e. make a full
backup), use \fI/dev/null\fR here. The file \fI/dev/null\fR is handled
specially by \fBrdup\fR: if detected no new file list is written. 
.PP
The
.IR DIRS/FILES
can be specified multiple times. These are the directories and files
you want to backup. If omitted it defaults to the current directory "." .
.PP
If the \fI\-N timestamp\fR option is not given, all paths found are
printed. Only when a \fI\-N timestamp\fR file is given can times be
compared and an incremental output generated. 
.PP
\fBrdup\fR prints a filelist to standard output. 
Subsequent programs in a pipe line can be used to actually
implement to backup scheme. 
After a run
a new FILELIST is written. No warning is given when FILELIST is an
existing file, it just gets overwritten by \fBrdup\fR. New runs will
print out only those files that have actually changed or are removed
since the last run, thereby making incremental backups possible.
.PP
Files are checked for changes by comparing the c\-time (change time),
if this time is NEWER than the c\-time of timestamp file the pathname is printed
to standard output. When files are removed they are also printed to 
standard output, but they are prefixed with a '-'. See 
.B FORMAT
below. The default format \fBrdup\fR uses is: "%p%T %b %t %u %U %g %G %l %s\\n%n%C"

Note, that \fBrdup\fR also supports hashing of files, this makes it
possible to check the local hash with the hash of the backed up file.
.PP
All errors are written to standard error.
If the directory or file does not exist, they are skipped and a
warning is emitted.
.PP
The general idea is to be very UNIX like and create a bunch of simple programs
which each do a their specific thing very well. With \fBrdup\fR and a
small shell script (50 lines) one can implement encrypted and compressed
backups.

As \fBrdup\fR doesn't backup anything, the backup policy; what you
backup, how you backup, how often and how you restore; is all left
to the scripts and your imagination. To kick start your imagination
see \fIrdup-tr(1)\fR, \fIrdup-up(1)\fR and maybe \fIrdup-backups\fR.

.SH OPTIONS
.TP
.B \-P\fI command\fR
Filter all output through \fIcommand\fR, multiple \-P's can be used.

Due to the nature of pipes in Unix, this pipeline is recreated for
every file processed. Also see 'Child Processes' below.
.TP
.B \-F format
Specify a printf-style format to use. See \fBFORMAT\fR below. 
.TP
.B \-N timestamp
use the c_time of file \fBtimestamp\fR as the timestamp to decide what to
include in the incremental backup list. If \fBtimestamp\fR does not exist
a full dump is performed. 
.TP
.B \-M timestamp
As \-N, but look at the m_time of timestamp.
.TP
.B \-R
Reverse the output of \fBrdup\fR. Tools accepting this ouput must
create leading directory as they see them. This option allows a script --
running as a normal user -- to put files in a directory which could have
0600 as its permission.
.TP
.B \-E file
The file named 'file' contains a list of Perl-compatible regular
expressions (PCRE), one per line, that \fBrdup\fR will use to 
\fIexclude\fR names. A '#' at the start of the line can be used to signal a comment.
Empty lines are ignored. 

If a directory is excluded, \fBrdup\fR won't descend in that directory,
so all files in that directory are also excluded.

The directories leading up to the directory to be backed up can not
be excluded. If you use a command line like: 

.RS
        rdup /dev/null /home/miekg/bin
.RE
.RS

The directories '/home', '/home/miekg', '/home/miekg/bin' are always printed.

If you want to exclude the file '/home/miekg/blaat' you need to add
the following regular expression: '/home/miekg/blaat'. 

If you want
to exclude all .mozilla/cache directories of all users you can 
use '/home/.*/.mozilla/cache/.*'. This doesn't exclude the directory
itself and I'm assuming that the users' home directories are found
under '/home'.

Also note that \fBrdup\fR does not print directories with a
trailing slash.
.RE

.TP
.B \-a
Restore the original access times on files and directories.
.TP
.B \-n
Don't honor \fI.nobackup\fR files. Normally if such a file is found the
directory and all files containing it, are not 
printed to standard output. Now they are.
.TP
.B \-u
Output files with a \fI._rdup_.\fR prefix just as normal files. When this option
is not given these files are skipped by \fIrdup\fR.
.TP
.B \-r
Only print removed files; entries that start with a `\-'. This 
option unsets \-m.
.TP
.B \-m
Only print modified/new files; entries that start with a `+'. This
option unsets \-r.
.TP
.B \-v
Be more verbose. When used each path will also be
printed to standard error. 
.TP
.B \-s size
Don't output files larger than \fBsize\fR bytes.
This can be used to limit the amount of data to be transferred when doing a remote backup.
This option \fIonly\fR applies to files.
.TP
.B \-x
Stay on the local filesystem.
.TP
.B \-V
Print rdup's version.
.TP
.B \-h
Give an overview of the options.

.SS Child Processes (-P flag)
When creating output you might also want to 'pipe' the contents
of \fIeach\fR file through a number of commands, say a compression and
encryption utility. Note that this is different than compressing the
\fIentire\fR archive as GNU tar allows by using the \-z option.
So this is where \fBrdup\fR comes in. It allows you to create
a normal archive in which \fIeach\fR file is encrypted (or compressed.
reversed or whatever). \fBrdup\fR does this by forking child
processes which transform the content.

If one of
the forked children returns an exit code other than zero (0), it
is assumed the whole conversion process failed. In that case \fBrdup\fR
terminates.

As said \fBrdup\fR works by forking off a number of child processes (those
commands named with the \-P option(s)), interconnecting these
with pipes. The current file is connected to the first child.
The output created by these child processes is
captured by the parent (\fBrdup\fR).
The contents is then written to standard output in an archive format. 
As a picture says more
than a thousand words here is an ASCII image of the process:

.RS

            +--- ...   (stdout)    ... ----> archive
           /
       rdup  <--- ...   ... <----+
                                 |
   loop #files                   |
                                 |
  file ---> cmd1 | cmd2 | ...| cmdN

.RE


.SH BACKUPS
With:
.RS
 rm -f timestamp && rdup -N timestamp LIST DIR
.RE

A full-dump filelist is printed to standard output. And with:

.RS
 rdup -N timestamp LIST DIR
.RE

An incremental dump filelist is printed. The file \fItimestamp\fR
is used to save the exact time of rdup's run. The file \fILIST\fR is
used to calculate the correct incremental dump list, this is needed
for files that are removed, or have a different type.

.SH NON-ROOT BACKUPS
If backups are made by a non-root user or on a filesystem that does
not implement/allow \fIchown\fR (think sshfs mounted by an ordinary user),
\fIrdup\fR creates a seperate file which stores the correct user and group
information. If the file's name is \fIfoobar\fR a new file called
\fI._rdup_.foobar\fR is created in the same directory. This file contains
one line, for instance:

.RS
    root:0/root:0
.RE

That tells that the \fIactual\fR ownership should be root:root. For directories
a \fI._rdup_.\fR file is created inside the current directory.

.SH FORMAT
The default format \fBrdup\fR uses is: "%p%T %b %t %u %U %g %G %l %s\\n%n%C"
.PP
The following escape sequences are understood by \fBrdup\fR:

.BR
        'p': '+' if file is new/modified, '-' if removed
.BR
        'b': permission bits from lstat(2), octal in four digits
.BR
        'm': the file mode bits, st_mode from lstat(2), decimal digits
.BR
        'u': uid
.BR
        'U': username
.BR
        'g': gid
.BR
        'G': groupname
.BR
        'l': path name length
.BR
        's': original file size, but see 
.B CAVEATS 
.BR
        'n': path name
.BR
        'N': path name, but in case of a soft- or hardlink \fIonly\fR
the link name
.BR
        't': time of modification (seconds from epoch)
.BR
        'H': the SHA1 hash of the file, all zeros ("0") for all other types
.BR
        'T': file type
.RS
       \fB-\fR normal file, \fBl\fR symlink, \fBh\fR hardlink, \fBd\fR directory,
       \fBc\fR character device, \fBb\fR block device, \fBp\fR named pipe
       and \fBs\fR socket
.RE
        'C': the content of the file (none for all other types)
.PP
To delimit the output of \fBrdup\fR with NULLs you can use '\\0' in the
format string.

Any file content is written in a block/chunk based manner. The
last block is signaled with a null block. A block start entry is 
ASCII and is formatted as follows: VVBLOCKBBBBB\\n .
Where 'VV' is the version, currently at '01', then the literal string 'BLOCK'
and then the amount of bytes (BBBBB), typical '08192'. And then a newline.
This look like this:

.RS
 01BLOCK08192
 <START OF THE FIRST 8192 BYTES>01BLOCK00015
 <ANOTHER 15 BYTES>01BLOCK00000
.RE

A byte count of zero signals a stop block.

.SH FILELIST
\fBrdup\fR writes the (internal) FILELIST in the following format:

.RS
MODE DEV INODE LINK UID GID PATH_SIZE FILE_SIZE PATH
.RE

.PP
Where MODE is the st_mode from stat(2), DEV is the dev id as returned by
the stat call and INODE is the inode number - \fBrdup\fR needs this info
to decide if a directory is renamed. LINK is equal 
to 'h' for hardlinks, 'l' for symlinks and otherwise it is '*'. 
UID and GID are the numeric user and group id of
the file. PATH_SIZE is the length of PATH. FILE_SIZE the file size.
And finally PATH is the path of the file. 

A typical example is: 

.RS
16893 2050 32085 * 1000 1000 30 4096 /home/miekg/git/rdup/.git/logs
.RE

.SH OUTPUT FORMAT 
The output generated by \fBrdup\fR is formatted like:

.RS
 +|-TYPE BITS MTIME UID USER GID GROUP PATH_SIZE FILE_SIZE\\n
 PATH FILE_CONTENTS
.RE

.PP
This makes it possible possible for a remote shell script to receive the
actual file contetns and make a backup.
.PP
For directories: the FILE_SIZE is zero and no content is printed. Thus:

.RS
 +d 0755 1260243445 1000 miekg 1000 miekg 11 0\\n
 \fB/home/miekg\fR
.RE

For regular files the following is a sample output:

.RS
 +- 0644 1260243445 1000 miekg 1000 miekg 32 6\\n
 \fB/home/miekg/svn/rdup/trunk/aaa/a\fR01BLOCK00006\\n
 \fIhello\fR\\n
 01BLOCK00000\\n
.RE

Where \fBaaa/a\fR is a regular file containing the word 'hello\\n'

.SS CAVEATS
Soft- and hardlinks are handled differently when using %n, if you don't
like this behavior use %N.
The PATH name is generated from the link's name and its target. A symlink like

.RS
 /home/bin/blaat -> /home/bin/bliep
.RE

is printed as '/home/bin/blaat -> /home/bin/bliep'. The PATH_SIZE
is modified accordingly, where '\fB -> \fR' (4 characters) is also counted.
The FILE_SIZE is not needed for soft- or hardlinks, so it is set the
length of the link's name -- the part left of the ' ->', in this case the 
length of '/home/bin/blaat'.

If rdup encounters a hardlink it is handled in the same way, but the
output type is set to 'h' instead of 'l'. A hardlink is only detected
if rdup finds a file with the same inode and device number as a previous
one, i.e. such hardlinks must be contained in your backup.

Again note: with '%N' only the link's name is printed. The FILE_SIZE is
\fIstill\fR set to the length of the link's name.

.SS Device Files
For devices the size field (%s) is changed to hold the major,minor number of
the device. So if a major number is 8 and the minor number is 0 (under
Linux this is /dev/sda), its size will be \fB8,0\fR. The numbers are
only separated with a comma `,'.

.SS Symlinks
You will probably think \fBrdup\fR will descend into the directory the
symbolic link points to. This is not what actually happens, \fBrdup\fR will
print any directories leading up to the symlink and will \fInot\fR
descend into the directory. GNU tar works the same.

.SH EXIT CODE
\fBrdup\fR return a zero exit code on success, otherwise 1 is returned.
\fRrdup\fR will abort if a file can not be concatenated, if a regular
expression can not be compiled or if a signal is received.

.SH EXAMPLES
The next set of examples will all make a full dump -- because of the use
of /dev/null. See \fIrdup-tr(1)\fR for more advanced examples.

.SS rdup (mirroring)
Backup:
.RS
 rdup /dev/null ~/bin | rdup-up -t /shared/backup          
.RE
Restore:
.RS
rdup /dev/null /shared/backup | rdup-up -t /tmp/restore
.RE
or
.RS
cp -rap /shared/backup /tmp/restore
.RE

.SS rdup (archiving)
Backup:
.RS
rdup /dev/null ~/bin > my-archive.rdup
.RE
Restore:
.RS
rdup-up -t /tmp/restore < my-archive.rdup
.RE

.SS cpio
Backup:
.RS
rdup -R -F '%N\\n' /dev/null ~/bin | cpio -o -Hcrc > my-archive.cpio          
.RE
Restore:
.RS
cpio -i -d -Hcrc < my-archive.cpio          
.RE

.SS tar
Backup:
.RS
rdup -F '%N\\n' /dev/null ~/bin | tar c -f my-archive.tar -T - --no-recursion
.RE
Restore:
.RS
tar x -f my-archive.tar
.RE

.SH AUTHOR
Written by Miek Gieben. 

.SH REPORTING BUGS
Report bugs to <miek@miek.nl>. 

.SH SEE ALSO
http:/www.miek.nl/projects/rdup is the main site of rdup. Also see
rdup-tr(1), rdup-up(1) and rdup-backups(7).

.SH COPYRIGHT
Copyright (C) 2005-2011 Miek Gieben. This is free software. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
PURPOSE.
.PP
Licensed under the GPL version 3. See the file LICENSE in the source distribution
of rdup.