OpenBox

An Open Sourced DropBox Clone

OpenBox is a lightweight, distributed cloud storage application inspired by Dropbox, written in Go. It demonstrates the fundamental principles of distributed systems, such as consistency, fault tolerance, and scalability, through its simple and modular design. OpenBox allows users to store, retrieve, and synchronize files seamlessly across multiple clients, simulating the core functionalities of a file-sharing and cloud storage service. OpenBox encapsulates the following principles:

• File Synchronization: Synchronize files between local directories and the cloud storage seamlessly.

• Distributed Architecture: Built with a modular structure consisting of clients, servers, and metadata management.

• Fault Tolerance: Ensures data reliability and availability even in the event of server failures.

• Version Control: Maintains a version history for files to handle conflicts and data consistency.

Usage

1. Run your server using this:

go run cmd/SurfstoreServerExec/main.go -s <service> -p <port> -l -d (BlockStoreAddr*)

Here, service should be one of three values: meta, block, or both. This is used to specify the service provided by the server. port defines the port number that the server listens to (default=8080). -l configures the server to only listen on localhost. -d configures the server to output log statements. Lastly, (BlockStoreAddr*) is the BlockStore address that the server is configured with. If service=both then the BlockStoreAddr should be the ip:port of this server.

2. Run your client using this:

go run cmd/SurfstoreClientExec/main.go -d <meta_addr:port> <base_dir> <block_size>

Examples:

go run cmd/SurfstoreServerExec/main.go -s both -p 8081 -l localhost:8081

This starts a server that listens only to localhost on port 8081 and services both the BlockStore and MetaStore interface.

Run the commands below on separate terminals (or nodes)
> go run cmd/SurfstoreServerExec/main.go -s block -p 8081 -l
> go run cmd/SurfstoreServerExec/main.go -s meta -l localhost:8081

The first line starts a server that services only the BlockStore interface and listens only to localhost on port 8081. The second line starts a server that services only the MetaStore interface, listens only to localhost on port 8080, and references the BlockStore we created as the underlying BlockStore. (Note: if these are on separate nodes, then you should use the public ip address and remove -l)

3. From a new terminal (or a new node), run the client using the script provided in the starter code (if using a new node, build using step 1 first). Use a base directory with some files in it.

> mkdir dataA
> cp ~/pic.jpg dataA/
> go run cmd/SurfstoreClientExec/main.go server_addr:port dataA 4096

This would sync pic.jpg to the server hosted on server_addr:port, using dataA as the base directory, with a block size of 4096 bytes.

4. From another terminal (or a new node), run the client to sync with the server. (if using a new node, build using step 1 first)

> mkdir dataB
> go run cmd/SurfstoreClientExec/main.go server_addr:port dataB 4096
> ls dataB/
pic.jpg index.txt

We observe that pic.jpg has been synced to this client.

Makefile

The Makefile for running both BlockStore and RaftStore Servers are provided below:

1. Run both BlockStore and RaftStore servers (listens to localhost on port 8081):

make run-both

2. Run BlockStore server (listens to localhost on port 8081):

make run-blockstore

3. Run RaftSurfstore server:

$ make IDX=0 run-raft

‍