Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/tpunder/fm-sbt-s3-resolver
SBT Plugin that adds support for resolving and publishing using Amazon S3
https://github.com/tpunder/fm-sbt-s3-resolver
s3 sbt-plugin scala
Last synced: 1 day ago
JSON representation
SBT Plugin that adds support for resolving and publishing using Amazon S3
- Host: GitHub
- URL: https://github.com/tpunder/fm-sbt-s3-resolver
- Owner: tpunder
- License: apache-2.0
- Created: 2014-05-05T16:31:11.000Z (almost 11 years ago)
- Default Branch: master
- Last Pushed: 2023-12-07T18:49:50.000Z (about 1 year ago)
- Last Synced: 2024-06-21T06:39:43.432Z (8 months ago)
- Topics: s3, sbt-plugin, scala
- Language: Scala
- Homepage:
- Size: 146 KB
- Stars: 162
- Watchers: 10
- Forks: 38
- Open Issues: 14
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# FM SBT S3 Resolver
[](https://github.com/tpunder/fm-sbt-s3-resolver/actions/workflows/build.yml) [](https://index.scala-lang.org/tpunder/fm-sbt-s3-resolver/fm-sbt-s3-resolver)
This SBT plugin adds support for using Amazon S3 for resolving and publishing using s3:// urls.
**Table of Contents** *generated with [DocToc](https://github.com/thlorenz/doctoc)*
- [SBT 1.X Support](#sbt-1x-support)
- [SBT 1.3 Support](#sbt-13-support)
- [SBT 1.1 Support](#sbt-11-support)
- [SBT 1.0 Support](#sbt-10-support)
- [Examples](#examples)
- [Resolving Dependencies via S3](#resolving-dependencies-via-s3)
- [Publishing to S3](#publishing-to-s3)
- [Valid s3:// URL Formats](#valid-s3-url-formats)
- [Usage](#usage)
- [Add this to your project/plugins.sbt file:](#add-this-to-your-projectpluginssbt-file)
- [S3 Credentials](#s3-credentials)
- [Bucket Specific Environment Variables](#bucket-specific-environment-variables)
- [Bucket Specific Java System Properties](#bucket-specific-java-system-properties)
- [Bucket Specific Property Files](#bucket-specific-property-files)
- [Environment Variables](#environment-variables)
- [Java System Properties](#java-system-properties)
- [Property File](#property-file)
- [Custom S3 Credentials](#custom-s3-credentials)
- [IAM Policy Examples](#iam-policy-examples)
- [Read/Write Policy (for publishing)](#readwrite-policy-for-publishing)
- [Read-Only Policy](#read-only-policy)
- [Releases Read-Only, Snapshots Read/Write](#releases-read-only-snapshots-readwrite)
- [IAM Role Policy Examples](#iam-role-policy-examples)
- [S3 Server-Side Encryption](#s3-server-side-encryption)
- [Maintainer](#maintainer)
- [Copyright](#copyright)
- [License](#license)
## SBT 1.X Support
Recent versions of this plugin should work fine on all SBT 1.X versions. If you run into problems please open up an Issue.
## SBT 1.3 Support
SBT 1.3 support is available using version `>= 0.19.0`:
```scala
addSbtPlugin("com.frugalmechanic" % "fm-sbt-s3-resolver" % "0.20.0")
```
## SBT 1.1 Support
SBT 1.1 support is available using version `>= 0.14.0`:
## SBT 1.0 Support
Note: **You need to use at least SBT 1.0.4** for this plugin to work with SBT 1.0 due to
https://github.com/sbt/librarymanagement/issues/175 which was fixed in this pull
request: https://github.com/sbt/librarymanagement/pull/183
## Examples
### Resolving Dependencies via S3
Maven Style:
```scala
resolvers += "FrugalMechanic Snapshots" at "s3://fm-sbt-s3-resolver-example-bucket/snapshots"
```
Ivy Style:
```scala
resolvers += Resolver.url("FrugalMechanic Snapshots", url("s3://fm-sbt-s3-resolver-example-bucket/snapshots"))(Resolver.ivyStylePatterns)
```
### Publishing to S3
Maven Style:
```scala
publishMavenStyle := true
publishTo := Some("FrugalMechanic Snapshots" at "s3://fm-sbt-s3-resolver-example-bucket/snapshots")
```
Ivy Style:
```scala
publishMavenStyle := false
publishTo := Some(Resolver.url("FrugalMechanic Snapshots", url("s3://fm-sbt-s3-resolver-example-bucket/snapshots"))(Resolver.ivyStylePatterns))
```
### Valid s3:// URL Formats
The examples above are using the [Static Website Using a Custom Domain](http://docs.aws.amazon.com/AmazonS3/latest/dev/website-hosting-custom-domain-walkthrough.html) functionality of S3.
These would also be equivalent (for the **fm-sbt-s3-resolver-example-bucket** bucket):
s3://s3-us-west-2.amazonaws.com/fm-sbt-s3-resolver-example-bucket/snapshots
s3://fm-sbt-s3-resolver-example-bucket.s3-us-west-2.amazonaws.com/snapshots
s3://fm-sbt-s3-resolver-example-bucket.s3.amazonaws.com/snapshots
s3://s3.amazonaws.com/fm-sbt-s3-resolver-example-bucket/snapshots
All of these forms should work:
s3://[BUCKET]/[OPTIONAL_PATH]
s3://s3.amazonaws.com/[BUCKET]/[OPTIONAL_PATH]
s3://[BUCKET].s3.amazonaws.com/[OPTIONAL_PATH]
s3://s3-[REGION].amazonaws.com/[BUCKET]/[OPTIONAL_PATH]
s3://[BUCKET].s3-[REGION].amazonaws.com/[OPTIONAL_PATH]
## Usage
### Add this to your project/plugins.sbt file:
```scala
addSbtPlugin("com.frugalmechanic" % "fm-sbt-s3-resolver" % "0.20.0")
```
### S3 Credentials
S3 Credentials are checked **in the following places and _order_** (e.g. bucket specific settings (\~/.sbt/.<bucket_name>_s3credentials) get resolved before global settings (\~/.sbt/.s3credentials)):
**Note: I think this logic has changed a little bit. See the S3URLHandler.defaultCredentialsProviderChain for the current implementation: https://github.com/frugalmechanic/fm-sbt-s3-resolver/blob/master/src/main/scala/fm/sbt/S3URLHandler.scala#L166**
#### Bucket Specific Environment Variables
AWS_ACCESS_KEY_ID_ -or- _AWS_ACCESS_KEY_ID
AWS_SECRET_KEY_ -or- _AWS_SECRET_KEY
**NOTE** - The following transforms are applied to the bucket name before looking up the environment variable:
1. The name is upper-cased
2. Dots (.) and dashes (-) are replaced with an underscore (_)
3. Everything other than A-Z, 0-9, and underscores are removed.
Example:
The bucket name "fm-sbt-s3-resolver-example-bucket" becomes "MAVEN\_FRUGALMECHANIC\_COM":
```shell
AWS_ACCESS_KEY_ID_MAVEN_FRUGALMECHANIC_COM="XXXXXX" AWS_SECRET_KEY_MAVEN_FRUGALMECHANIC_COM="XXXXXX" sbt
```
#### Bucket Specific Java System Properties
```shell
-Daws.accessKeyId.=XXXXXX -Daws.secretKey.=XXXXXX
-D.aws.accessKeyId=XXXXXX -D.aws.secretKey=XXXXXX
```
Example:
```shell
SBT_OPTS="-Daws.accessKeyId.fm-sbt-s3-resolver-example-bucket=XXXXXX -Daws.secretKey.fm-sbt-s3-resolver-example-bucket=XXXXXX" sbt
```
#### Bucket Specific Property Files
```shell
~/.sbt/._s3credentials
~/.sbt/.s3credentials_
```
#### Environment Variables
AWS_ACCESS_KEY_ID (or AWS_ACCESS_KEY)
AWS_SECRET_KEY (or AWS_SECRET_ACCESS_KEY)
AWS_ROLE_ARN
Example:
```shell
// Basic Credentials
AWS_ACCESS_KEY_ID="XXXXXX" AWS_SECRET_KEY="XXXXXX" sbt
// IAM Role Credentials
AWS_ACCESS_KEY_ID="XXXXXX" AWS_SECRET_KEY="XXXXXX" AWS_ROLE_ARN="arn:aws:iam::123456789012:role/RoleName" sbt
```
#### Java System Properties
// Basic Credentials
-Daws.accessKeyId=XXXXXX -Daws.secretKey=XXXXXX
// IAM Role
-Daws.accessKeyId=XXXXXX -Daws.secretKey=XXXXXX -Daws.arnRole=arn:aws:iam::123456789012:role/RoleName
Example:
```shell
// Basic Credentials
SBT_OPTS="-Daws.accessKeyId=XXXXXX -Daws.secretKey=XXXXXX" sbt
// IAM Role Credentials
SBT_OPTS="-Daws.accessKeyId=XXXXXX -Daws.secretKey=XXXXXX -Daws.arnRole=arn:aws:iam::123456789012:role/RoleName" sbt
```
#### Property File
```shell
~/.sbt/.s3credentials
```
The property files should have the following format:
```ini
accessKey = XXXXXXXXXX
secretKey = XXXXXXXXXX
// Optional IAM Role
roleArn = arn:aws:iam::123456789012:role/RoleName
```
### Custom S3 Credentials
If the default credential providers do not work for you then you can specify your own AWSCredentialsProvider using the `s3CredentialsProvider` SettingKey in your `build.sbt` file:
```scala
import com.amazonaws.auth.{AWSCredentialsProviderChain, DefaultAWSCredentialsProviderChain}
import com.amazonaws.auth.profile.ProfileCredentialsProvider
s3CredentialsProvider := { (bucket: String) =>
new AWSCredentialsProviderChain(
new ProfileCredentialsProvider("my_profile"),
DefaultAWSCredentialsProviderChain.getInstance()
)
}
```
If you are really lazy and want to provide static credentials using this in your `build.sbt` file will work:
```scala
import com.amazonaws.auth.{AWSStaticCredentialsProvider, BasicAWSCredentials}
s3CredentialsProvider := { (bucket: String) =>
new AWSStaticCredentialsProvider(new BasicAWSCredentials("your_accessKey", "your_secretKey"))
}
```
## IAM Policy Examples
I recommend that you create IAM Credentials for reading/writing your Maven S3 Bucket. Here are some examples for our **fm-sbt-s3-resolver-example-bucket** bucket:
### Read/Write Policy (for publishing)
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": ["s3:GetBucketLocation"],
"Resource": "arn:aws:s3:::*"
},
{
"Effect": "Allow",
"Action": ["s3:ListBucket"],
"Resource": ["arn:aws:s3:::fm-sbt-s3-resolver-example-bucket"]
},
{
"Effect": "Allow",
"Action": ["s3:DeleteObject","s3:GetObject","s3:PutObject"],
"Resource": ["arn:aws:s3:::fm-sbt-s3-resolver-example-bucket/*"]
}
]
}
### Read-Only Policy
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": ["s3:GetBucketLocation"],
"Resource": "arn:aws:s3:::*"
},
{
"Effect": "Allow",
"Action": ["s3:ListBucket"],
"Resource": ["arn:aws:s3:::fm-sbt-s3-resolver-example-bucket"]
},
{
"Effect": "Allow",
"Action": ["s3:GetObject"],
"Resource": ["arn:aws:s3:::fm-sbt-s3-resolver-example-bucket/*"]
}
]
}
### Releases Read-Only, Snapshots Read/Write
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": ["s3:GetBucketLocation"],
"Resource": "arn:aws:s3:::*"
},
{
"Effect": "Allow",
"Action": ["s3:ListBucket"],
"Resource": ["arn:aws:s3:::fm-sbt-s3-resolver-example-bucket"]
},
{
"Effect": "Allow",
"Action": ["s3:GetObject"],
"Resource": ["arn:aws:s3:::fm-sbt-s3-resolver-example-bucket/releases/*"]
},
{
"Effect": "Allow",
"Action": ["s3:DeleteObject","s3:GetObject","s3:PutObject"],
"Resource": ["arn:aws:s3:::fm-sbt-s3-resolver-example-bucket/snapshots/*"]
}
]
}
## IAM Role Policy Examples
This is a simple example where a Host AWS Account, can create a Role with permissions for a Client AWS Account to access the Host maven bucket.
1. Host AWS Account, creates an IAM Role named "ClientAccessRole" with policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::[Client AWS Account Id]:user/[Client User Name]"
},
"Action": "sts:AssumeRole"
}
]
}
2. Associate the proper [IAM Policy Examples](#iam) to the Host Role
3. Client AWS Account needs to create an AWS IAM User [Client User Name] and associated a policy to gives it permissions to AssumeRole from the Host AWS Account:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "sts:AssumeRole",
"Resource": "arn:aws:iam::[Host AWS Account Id]:role/ClientAccessRole"
}
]
}
## S3 Server-Side Encryption
S3 supports server side encryption.
The plugin will automatically detect if it needs to ask S3 to use SSE, based on the policies you have on your bucket. If
your bucket denies `PutObject` requests that aren't using SSE, the plugin will include the SSE header in future requests.
To make use of SSE, configure your bucket to enforce the SSE header for `PutObject` requests.
Example:
{
"Version": "2012-10-17",
"Id": "PutObjPolicy",
"Statement": [
{
"Sid": "DenyIncorrectEncryptionHeader",
"Effect": "Deny",
"Principal": "*",
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::YOUR_BUCKET_HERE/*",
"Condition": {
"StringNotEquals": {
"s3:x-amz-server-side-encryption": "AES256"
}
}
},
{
"Sid": "DenyUnEncryptedObjectUploads",
"Effect": "Deny",
"Principal": "*",
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::YOUR_BUCKET_HERE/*",
"Condition": {
"Null": {
"s3:x-amz-server-side-encryption": "true"
}
}
}
]
}
## Maintainer
Tim Underwood (GitHub, LinkedIn, Twitter)
## License
[Apache License, Version 2.0](http://www.apache.org/licenses/LICENSE-2.0.txt)