Skip to content

A simple synchronization algorithm avoiding sending of redundant data.

License

Notifications You must be signed in to change notification settings

spinlockirqsave/rsyncme

Repository files navigation

/*@file         README
* @brief        Introduction to RSYNCME.
* @author       Piotr Gregor <[email protected]>
* @date         24 Jan 2016 06:16 PM
* @copyright    LGPLv2.1 */


Content
1. OVERVIEW
2. DOWNLOAD
3. DIRECTORY STRUCTURE
4. BUILD , TEST, INSTALLATION
5. TESTS
6. USAGE
7. DEBUGGING
8. CLEAN



1. OVERVIEW

RSYNCME is open source tool for easy and efficient synchronization of data
locally (on the same PC) and remotely (PC <-> PC) over TCP connection.
The main idea is to avoid sending of redundant data, that is these bytes
from file which didn't change from last sync. The bytes which need to be sent
are prepared without a need of having the remote file available locally (a priori).
We hope the concept to prove useful on low bandwidth links, when the time needed
to transmit data overwhelms the time that alternatively will be spent
by endpoints on computing differences and performing data reconstruction.

The synchronization algorithm is derived from a well known rsync algorithm
but we offer different options to the algorithm and believe to have better
and simpler implementation.



2. DOWNLOAD

Git repository:
    git clone --recursive https://bitbucket.org/rsyncmedev/rsyncme.git
Binary packages:
    https://bitbucket.org/rsyncmedev/rsyncme/downloads?tab=downloads
Versions:
    https://bitbucket.org/rsyncmedev/rsyncme/downloads?tab=tags



3. DIRECTORY STRUCTURE

rsyncme/		<-- this directory (ROOT)
    - README
        This file
    - USER_GUIDE.txt
        A brief tutorial
    - LICENSE
    - include/
        - twlist/
            Submodule (implementation of list, queue and hashtable)
    - src/
    - build/
        - debug/
        - release/
    - test/
        - include/
        - src/
        - build/
            -debug/
                -log/
                    Test logs (debug mode) are written here
            -release/
                -log/
                    Test logs (release mode) are written here
    - install_deps.sh
                    Script for installation of dependencies (see 4.1)

/usr/
    - local/
        - rsyncme/
            - log/
				Server logs are written here



4. BUILD , TEST, INSTALLATION

There are few targets available: rsyncme_d, rsyncme, test executables.
Only commandline utility "rsyncme" is needed for local synchronization
Installation scripts are available for out of the box installation.
on Linux Debian and Ubuntu - there is no support for other platforms
at this moment. Linux Debian is suggested.

4.1 Dependencies

All sources depend on libuuid, so please be sure it is installed on your system
or install it using our script (this will install cmocka support too, which is required
if you wish to run tests).
Our code depends on twlist from Piotr Gregor's github repository, so be sure you did
git checkout with --recursive option:
    git clone --recursive https://bitbucket.org/rsyncmedev/rsyncme.git

4.1.1 Installing libuuid:

    A. Manually
        apt-get install uuid uuid-dev
    B. Using our script
        sh install_deps.sh cmocka	(give name of folder where to install cmocka,
                                    e.g. "cmocka", you may need sudo to install
                                    packages)
        
Our tests depend on cmocka, which depends on cmake, so please be sure
to have them installed first if you want to run tests (you can do it manually
following below instructions or run our install_deps.sh script).

4.1.2 Installing cmocka:

    A. Using script:
        sh install_deps.sh cmocka	(give name of folder where to install cmocka,
                                    e.g. "cmocka", you may need sudo to install
                                    packages)

    B. To install manually:
        First, install cmake:
            sudo apt-get install cmake
        Then install cmocka:
            1. Download https://cmocka.org/files/1.0/cmocka-1.0.1.tar.xz
            2. Follow instructions from INSTALL file

4.2 Default release build (be sure you did git checkout with --recursive option):

    cd src
    make    (will build everything [apart from tests] in release mode,
            same as "make all" and "make release")

4.3 Default debug build (be sure you did git checkout with --recursive option):

    cd src
    make debug

4.4 Installation

    make install    (you may need to run this with sudo depending on your environment)



5. TESTS

Please be sure dependencies are installed (see 4.1)

5.1. To build tests:

Make sure cmocka support is installed (see 4.1)

	cd src
	make test       ("make test-debug" for debug)

5.2 To run tests:

	cd src
	make test-check     ("make test-check-debug" for debug)
        Make will execute all cmocka test-suites.
        Results of each are written in folder ROOT/test/build/release/log
        (ROOT/test/build/debug/log if debug tests are executed)
        Short summary of each executed test suite is saved
        in rsyncme_test_<#>_<timestamp>.log file. Detailed log is saved
        in rsyncme_test_<#>_<timestamp>.err for each of the test suites.
        If you would like to inspect intermediate files created during each
        test suite <#> run, please change RM_TEST_<#>_DELETE_FILES
        definition in header ROOT/test/include/test_rm<#>.h to 0.



6. USAGE

    Please read USER_GUIDE.txt.



7. DEBUGGING

Debugging should be done from ROOT/test/src folder.
E.g:

    piotr@world:~/projects/rsyncme/test/src$ gdb ../build/debug/test_rm_main1



8. CLEAN

    To clean everything:

	    cd src
	    make clean-all

    To clean rsyncme_d (daemon's executable) and rsyncme (commandline executable):

	    cd src
	    make clean

    To clean tests (but not test logs):

	    cd src
	    make test-clean

    To clean test logs:

	    cd src
	    make test-clean-logs

    To clean tests-debug build(but not test logs):

	    cd src
	    make test-clean-debug

    To clean test-debug logs:

	    cd src
	    make test-clean-logs-debug

    To clean test and test logs (release and debug):

	    cd src
	    make test-clean-all



Website:        http://rsyncme.org
Development:    https://bitbucket.org/rsyncmedev/rsyncme
Mirror:         https://github.com/spinlockirqsave/rsyncme
Contact:        [email protected]