https://github.com/namshi/s3-sidecar

A container that works as a volume sidecar within a k8s pod.
https://github.com/namshi/s3-sidecar

Last synced: 1 day ago
JSON representation

A container that works as a volume sidecar within a k8s pod.

• Host: GitHub
• URL: https://github.com/namshi/s3-sidecar
• Owner: namshi
• Created: 2017-03-08T07:47:52.000Z (over 7 years ago)
• Default Branch: master
• Last Pushed: 2021-03-31T06:30:25.000Z (over 3 years ago)
• Last Synced: 2024-04-14T12:15:20.371Z (7 months ago)
• Language: Shell
• Size: 8.79 KB
• Stars: 5
• Watchers: 35
• Forks: 3
• Open Issues: 1
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# s3-sidecar

A container downloads a compressed file from S3 and extract the files into a local directory `/data`

which can be a shared volume with other containers

## Installing and Running

You need to set the env variables and run it as contaner, and here are the required variables:

```

    AWS_ACCESS_KEY_ID: 'access key'

    AWS_SECRET_ACCESS_KEY: 'access secret'

    AWS_DEFAULT_REGION: 'aws region'

    AWS_BUCKET: 'bucket name'

```

Build the container `docker-compose build`

Then you can run it `docker-compose up`

#How it works

Once you run the container, it will start a script which will run in a loop every 3 seconds.

It will get the new file name we need to download from `CONFIG_PATH` as the last part after `/` and then it will check that the 

content of this file `/data/{file name}` is different from the file name we are downloading.

If they are different it will download the file from S3 and extract it in teh `/data/` directory.

It will create a file `/data/{file name}` contains the name of the extracted file,

then it creates a lock file `/data/update.lock`, this file can be used from other containers to clear their cache.

If the file name in `/data/{file name}` is the same as the `file name` we are downloading, it won't do anything.