Surfstore

Surfstore is a cloud-based file storage service(similar to Dropbox). Multiple clients can sync files from and to the cloud.

The SurfStore service is composed of two services:

• BlockStore: The content of each file in SurfStore is divided up into chunks, or blocks, each of which has a unique identifier. This service stores these blocks, and when given an identifier, retrieves and returns the appropriate block.

• MetadataStore: The MetadataStore service holds the mapping of filenames to hashes of these blocks (and versions).

This is the starter code for Module 3: Surfstore. Before you get started, make sure you understand the following 2 things about Go. (These will also be covered in class and in discussions)

1. Interfaces: They are named collections of method signatures. Here are some good resources to understand interfaces in Go: a. https://gobyexample.com/interfaces b. https://jordanorelli.com/post/32665860244/how-to-use-interfaces-in-go

2. RPC: You should know how to write RPC servers and clients in Go. The online documentation of the rpc package is a good resource.

Setup

You will need to setup your runtime environment variables so that you can build your code and also use the executables that will be generated.

1. If you are using a Mac, open ~/.bash_profile or if you are using a unix/linux machine, open ~/.bashrc. Then add the following:

export GOPATH=<path to starter code>
export PATH=$PATH:$GOPATH/bin

2. Run source ~/.bash_profile or source ~/.bashrc

Usage

1. Only after you have implemented all the methods and completed the Setup steps, run the build.sh script provided with the starter code. This should create 2 executables in the bin folder inside your starter code directory.

> ./build.sh
> ls bin
SurfstoreClientExec SurfstoreServerExec

2. Run your server using the script provided in the starter code.

./run-server.sh

3. From a new terminal (or a new node), run the client using the script provided in the starter code (if using a new node, build using step 1 first). Use a base directory with some files in it.

> mkdir dataA
> cp ~/pic.jpg dataA/ 
> ./run-client.sh server_addr:port dataA 4096

This would sync pic.jpg to the server hosted on server_addr:port, using dataA as the base directory, with a block size of 4096 bytes.

4. From another terminal (or a new node), run the client to sync with the server. (if using a new node, build using step 1 first)

> ls dataB/
> ./run-client.sh server_addr:port dataB 4096
> ls dataB/
pic.jpg index.txt

We observe that pic.jpg has been synced to this client.