BinCache is a system designed to chache compiled binaries on S3. It has resources for bash, ruby and chef.
BinCache tracks a single directory and a series of scripts. After each script is run, the contents of the directory is cached on s3. On sequential runs, the scripts are not re-executed, but instead the cache from S3 is downloaded and placed in the directory. If a script is changed, itself and all following scripts will be re-executed. BinCache does this by hashing the current script together with all previous scripts ran and storing the resulting cache in S3 with that hash as it’s name. By ensuring that scripts contain markers to introduce uniquness on different systems, you can make sure that you never have to wait twice for something to compile!
## add rubygems.org to your gem sources list if it is not already there gem sources -a https://rubygems.org ## install bincache gem install bincache
Make sure you set the environment variables with something like:
## S3 access key id export BINCACHE_S3_ACCESS_KEY=1234567890ABCDEFGHIJ ## S3 secret access key export BINCACHE_S3_SECRET_KEY=1234567890ABCDEFHIJKLMNOPQRSTUVWXYZ12345 ## S3 bucket export BINCACHE_S3_BUCKET=my_ec2_bucket ## prefix inside bucket to store caches export BINCACHE_S3_PREFIX=my_bincache/
## you may have to put rubygem bins in your path by doing something like.. export PATH=$HOME/.gem/ruby/1.8/bin:$PATH ## set the directory to cache dir=/tmp/bincache ## create a some sample scripts cat <<-EOF > script1 echo "I am in script one" > one EOF cat <<-EOF > script2 echo "I am in script two" > two EOF ## run bincache bincache $dir $script1 $script2
## make sure bincache is loaded require 'bincache' ## set the directory to cache dir = "/tmp/bincache" ## create two sample scripts script1 = <<EOS echo "I am in script one" > one EOS script2 = <<EOS echo "I am in script two" > two EOS ## create an empty list of scripts and add the sample script to it scripts = [] scripts << script1 scripts << script2 ## run bincache bincache = BinCache.new bincache.run_series(dir,scripts)
require 'bincache' bincache "script1" do action :run directory "/tmp/bincache" script <<-EOS echo "I am in script one" > one EOS end bincache "script2" do action :run directory "/tmp/bincache" script <<-EOS echo "I am in script two" > two EOS end
To tag a script with something unique to ensure that it is distingushible on multiple systems, you need to insert something unique into the comments. For example, you could do:
echo "I am showing you how to uniqueify your script" > example ## #{`uname -m`} ## #{`lsb_release -ds`
When this script is hashed, it will produce different values on different systems. With this technique, you can cache different binaries from different systems with the same code base!
Copyright © 2010 Martin Rhoads. See LICENSE for details.