https://github.com/kierse/distributed-app
eece513 distributed-app
https://github.com/kierse/distributed-app
distributed-computing distributed-file-system distributed-systems kotlin
Last synced: over 1 year ago
JSON representation
eece513 distributed-app
- Host: GitHub
- URL: https://github.com/kierse/distributed-app
- Owner: kierse
- License: mit
- Created: 2019-03-27T00:01:16.000Z (over 7 years ago)
- Default Branch: master
- Last Pushed: 2019-03-27T00:32:33.000Z (over 7 years ago)
- Last Synced: 2025-01-27T08:23:10.002Z (over 1 year ago)
- Topics: distributed-computing, distributed-file-system, distributed-systems, kotlin
- Language: Kotlin
- Size: 235 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# README #
Attention: for Windows users, replace `gradlew` with `gradlew.bat`
## How to Use ##
- Set up EC2 instance and pull from master
- Update instances to [Java 9](#markdown-header-to-install-java9) if required
- Create a "server.txt" file in the project root with Public DNS addresses of each EC2 instance
- Generate public/private ssh key (without password) and save to keys/server-to-server:
_Note: from the project root directory_
```
> ssh-keygen -t rsa -b 4096 -C "server-to-server"
```
- Initialize the instances with the following commands:
_Note: substitute first argument with path to actual pem file_
```
> ./script/deploymentScript /Users/kierse/.ssh/eece513_aws.pem --ssh-keys
```
- Run the following command to build and deploy:
_Note: substitute populate pem flag with path to actual file_
```
> ./gradlew bootstrapAws -Ppem=/Users/kierse/.ssh/eece513_aws.pem
```
- In any EC2 instance home directory, start a cluster node:
```
> java -jar distributed-app-1.0.jar
```
- On startup, the node will print out its join address. This address can be used by other nodes wishing to join its cluster:
_Note: replace 127.0.0.1 with the join address obtained in the previous step_
```
> java -jar distributed-app-1.0.jar 127.0.0.1
```
**_Warning: starting a node without the address of an existing cluster will effectively create a second cluster._**
### Run commands across cluster ###
_Note: substitute path to actual pem file_
```
> csshx/csshX --login ec2-user -ssh_args "-i ~/.ssh/eece513_aws.pem" --hosts servers.txt
```
## fs513 ##
fs513 is a command line script that can be used to interact with the distributed filesystem.
```
usage: ./fs513 [ ]
Actions:
put (add a local file to fs513 with the given fs513 name)
get (fetch a fs513 file to the local machine)
remove (delete a file from fs513)
grep [ ... ] (search the fs513 server logs)
locate (list all machines (name / id / IP address) of the servers that contain a copy of the file)
ls (list all files in fs513)
lshere (list all fs513 files stored on the local machine)
--help (print this message)
```
### Supported fs513 commands ###
##### add file #####
```
./fs513 put foo.txt foo/bar/baz
```
##### get file #####
```
./fs513 get foo/bar/baz foo.txt
```
##### remove file #####
```
./fs513 remove foo/bar/baz
```
##### grep server logs #####
```
./fs513 grep -i "no files to sync""
```
##### locate file #####
```
./fs513 locate foo/bar/baz
```
##### list files in filesystem #####
```
./fs513 ls
```
##### list files on current machine #####
```
./fs513 lshere
```
## Tests ##
#### Unit Tests ####
```
> ./gradlew test
```
#### Distributed Tests ####
```
> ./gradlew distributedTest
```
## UPGRADE FROM JAVA7/8 TO JAVA9 ###
### To install Java9 ###
```
> wget --no-cookies --no-check-certificate --header "Cookie: oraclelicense=accept-securebackup-cookie" http://download.oracle.com/otn-pub/java/jdk/9.0.4+11/c2514751926b4512b076cc82f959763f/jdk-9.0.4_linux-x64_bin.rpm
> sudo yum install jdk-9.0.4_linux-x64_bin.rpm
```
### To remove old Java versions ###
```
> sudo yum remove java-1.8.0-openjdk
> sudo yum remove java-1.7.0-openjdk
```