https://github.com/airbnb/sssp
Software distribution by way of S3 signed URLs
https://github.com/airbnb/sssp
Last synced: about 1 month ago
JSON representation
Software distribution by way of S3 signed URLs
- Host: GitHub
- URL: https://github.com/airbnb/sssp
- Owner: airbnb
- License: other
- Created: 2012-03-03T01:15:06.000Z (over 13 years ago)
- Default Branch: master
- Last Pushed: 2012-08-20T16:02:25.000Z (about 13 years ago)
- Last Synced: 2024-05-21T03:15:27.402Z (over 1 year ago)
- Language: Haskell
- Homepage:
- Size: 161 KB
- Stars: 47
- Watchers: 305
- Forks: 15
- Open Issues: 0
-
Metadata Files:
- Readme: README
- License: LICENSE
Awesome Lists containing this project
README
SYNOPSIS
sssp web?
DESCRIPTION
SSSP is an HTTP proxy for S3 that can generate short-lived, signed URLs
for stored objects. By providing a server separate from S3 that can be
placed behind an authenticating proxy or firewall, SSSP allows a vari-
ety of common security mechanisms to be used to limit access to S3
objects over HTTP while taking advantage of S3's considerable bandwidth
and parallelism.
Use-cases for SSSP include:
o sharing of large files within an organization,
o media service for public facing web applications,
o distribution of internal software.
SSSP supports configuration via environment variables or STDIN.
CONFIGURATION
These settings can be passed as environment variables or fed to the
server on STDIN in colon separated format.
# AWS Settings
AWS_ACCESS_KEY_ID = account access key
AWS_SECRET_ACCESS_KEY = secret
AWS_REGION = eu-west-1, classic, us-east-1, ...
# Storage settings
SSSP_BUCKET = DNS friendly bucket name
# Server settings
SSSP_CONN = : pair
PORT = port to connect to, on localhost
SSSP is fairly liberal when parsing STDIN. In fact, Bourne shell .rc
files, like the follow example, are parsed without error:
export SSSP_BUCKET=dist
export SSSP_CONN=*:6000
However, SSSP does not parse shell quotes (which one is unlikely to
need for the kinds of values involved, given their rigid formats).
REST INTERFACE
URLs in SSSP point to one of two objects: an item or a listing. Items
correspond to S3 objects; a GET retrieves a signed redirect to the
object. Listings are a sequence of URLs, in ascending order; a GET
retrieves the listing as a plaintext document, one URL per line.
GET http://sssp.io/p/a/t/h # Signed for the default time (10s).
A PUT to an item sets the item's content. DELETEs can be singular or
plural. A plural DELETE removes only the objects generated by a list-
ing.
URLs are divided syntactically in to listings and items. A URL ending
with a slash is always a listing.
GET http://sssp.io/dist # Signed redirect to an object called dist.
GET http://sssp.io/dist/ # Listing of items below the key `dist'.
To make it easier to work with versioned or timestamped assets, SSSP
supports the @hi and @lo meta-paths. These correspond to the names that
sort highest and lowest according to semantic version sort, where
non-digit chars serve to delimit arrays of numbers. For common forms of
dates, these have the same effect as ASCII sort. (ASCII sort may speci-
fied, as well; please the section WILDCARDS, below.)
GET http://sssp.io/dist/x/x-0.1.1.tgz
GET http://sssp.io/dist/x/x-0.1.4.tgz
GET http://sssp.io/dist/x/x-0.2.11.tgz
GET http://sssp.io/dist/x/x-0.2.9.tgz
# Retrieval with @hi and @lo.
GET http://sssp.io/dist/x/@hi -307-> http://sssp.io/dist/x/x-0.2.11.tgz
GET http://sssp.io/dist/x/@lo -307-> http://sssp.io/dist/x/x-0.1.1.tgz
Wildcards @hi and @lo used together with a count specify a set wild-
card; the result is a listing:
GET http://sssp.io/dist/x/@lo2 -200-> dist/x/x-0.1.1.tgz
dist/x/x-0.1.4.tgz
Counts are the natural numbers starting at 0.
A counted wildcard, like @hi2, can be suffixed with a tilde to form
it's complement -- so @hi2~ is everything but the highest two items.
This can be useful for bulk deletion of old/new things.
WILDCARDS
@hi.semver, @lo.semver
Key with highest or lowest version, according to a liberal-
ized form of "semantic versioning", where version components
are delimited by any non-digit characters.
@hi.ascii, @lo.ascii
Keys sorted ASCIIbetically, in the C locale (sorted purely by
byte value).
@hi, @lo
The default sort, which is semantic version sort.
EXAMPLES
# Start web application.
sssp < conf
# Start web application with some configuration provided by the environment.
export AWS_ACCESS_KEY_ID=...
export AWS_SECRET_ACCESS_KEY=...
sssp <