https://github.com/zio/zio-s3
An S3 client for ZIO
https://github.com/zio/zio-s3
Last synced: 6 months ago
JSON representation
An S3 client for ZIO
- Host: GitHub
- URL: https://github.com/zio/zio-s3
- Owner: zio
- License: apache-2.0
- Created: 2020-01-03T17:55:59.000Z (almost 6 years ago)
- Default Branch: series/2.x
- Last Pushed: 2024-10-10T18:47:40.000Z (about 1 year ago)
- Last Synced: 2025-03-30T19:07:04.414Z (7 months ago)
- Language: Scala
- Homepage: https://zio.dev/zio-s3/
- Size: 1.58 MB
- Stars: 44
- Watchers: 6
- Forks: 31
- Open Issues: 16
-
Metadata Files:
- Readme: README.md
- License: LICENSE
- Codeowners: .github/CODEOWNERS
Awesome Lists containing this project
README
[//]: # (This file was autogenerated using `zio-sbt-website` plugin via `sbt generateReadme` command.)
[//]: # (So please do not edit it manually. Instead, change "docs/index.md" file or sbt setting keys)
[//]: # (e.g. "readmeDocumentation" and "readmeSupport".)
# ZIO S3
[ZIO S3](https://github.com/zio/zio-s3) is a thin wrapper over S3 async client for ZIO.
[](https://github.com/zio/zio/wiki/Project-Stages)  [](https://oss.sonatype.org/content/repositories/releases/dev/zio/zio-s3_2.13/) [](https://oss.sonatype.org/content/repositories/snapshots/dev/zio/zio-s3_2.13/) [](https://javadoc.io/doc/dev.zio/zio-s3-docs_2.13) [](https://github.com/zio/zio-s3)
## Introduction
ZIO-S3 is a thin wrapper over the s3 async java client. It exposes the main operations of the s3 java client.
```scala
import software.amazon.awssdk.auth.credentials.AwsBasicCredentials
import zio.Chunk
import zio.s3._
import zio.stream.{ZSink, ZStream}
import software.amazon.awssdk.services.s3.model.S3Exception
// list all buckets available
listBuckets.provideLayer(
live("us-east-1", AwsBasicCredentials.create("accessKeyId", "secretAccessKey"))
)
// list all objects of all buckets
val l2: ZStream[S3, S3Exception, String] = (for {
bucket <- ZStream.fromIterableZIO(listBuckets)
obj <- listAllObjects(bucket.name)
} yield obj.bucketName + "/" + obj.key).provideLayer(
live("us-east-1", AwsBasicCredentials.create("accessKeyId", "secretAccessKey"))
)
```
All available s3 combinators and operations are available in the package object `zio.s3`, you only need to `import zio.s3._`
## Installation
In order to use this library, we need to add the following line in our `build.sbt` file:
```scala
libraryDependencies += "dev.zio" %% "zio-s3" % "0.4.3"
```
## Example 1
Let's try an example of creating a bucket and adding an object into it. To run this example, we need to run an instance of _Minio_ which is object storage compatible with S3:
```bash
docker run -p 9000:9000 -e MINIO_ROOT_USER=MyKey -e MINIO_ROOT_PASSWORD=MySecret minio/minio server --compat /data
```
In this example we create a bucket and then add a JSON object to it and then retrieve that:
```scala
import software.amazon.awssdk.auth.credentials.AwsBasicCredentials
import software.amazon.awssdk.regions.Region
import zio._
import zio.s3._
import zio.stream.{ZStream, ZPipeline}
import zio.Chunk
import java.net.URI
object ZIOS3Example extends ZIOAppDefault {
val myApp = for {
_ <- createBucket("docs")
json = Chunk.fromArray("""{ "id" : 1 , "name" : "A1" }""".getBytes)
_ <- putObject(
bucketName = "docs",
key = "doc1",
contentLength = json.length.toLong,
content = ZStream.fromChunk(json),
options = UploadOptions.fromContentType("application/json")
)
_ <- getObject("docs", "doc1")
.via(ZPipeline.utf8Decode)
.foreach(Console.printLine(_))
} yield ()
def run =
myApp
.provide(
live(
Region.CA_CENTRAL_1,
AwsBasicCredentials.create("MyKey", "MySecret"),
Some(URI.create("http://localhost:9000")),
forcePathStyle = Some(true) // Required for path-style S3 requests (MinIO by default uses them)
)
)
}
```
## Example 2
```scala
import software.amazon.awssdk.services.s3.model.S3Exception
import zio._
import zio.stream.{ ZSink, ZStream }
import zio.s3._
// upload
val json: Chunk[Byte] = Chunk.fromArray("""{ "id" : 1 , "name" : "A1" }""".getBytes)
val up: ZIO[S3, S3Exception, Unit] = putObject(
"bucket-1",
"user.json",
json.length.toLong,
ZStream.fromChunk(json),
UploadOptions.fromContentType("application/json")
)
// multipartUpload
import java.io.FileInputStream
import java.nio.file.Paths
val is = ZStream.fromInputStream(new FileInputStream(Paths.get("/my/path/to/myfile.zip").toFile))
val proc2: ZIO[S3, S3Exception, Unit] =
multipartUpload(
"bucket-1",
"upload/myfile.zip",
is,
MultipartUploadOptions.fromUploadOptions(UploadOptions.fromContentType("application/zip"))
)(4)
// download
import java.io.OutputStream
val os: OutputStream = ???
val proc3: ZIO[S3, Exception, Long] = getObject("bucket-1", "upload/myfile.zip").run(ZSink.fromOutputStream(os))
```
## Support any commands?
If you need a method which is not wrapped by the library, you can have access to underlying S3 client in a safe manner by using
```scala
import java.util.concurrent.CompletableFuture
import zio.s3._
import software.amazon.awssdk.services.s3.S3AsyncClient
def execute[T](f: S3AsyncClient => CompletableFuture[T])
```
## Documentation
Learn more on the [ZIO S3 homepage](https://zio.dev/zio-s3/)!
## Contributing
For the general guidelines, see ZIO [contributor's guide](https://zio.dev/about/contributing).
## Code of Conduct
See the [Code of Conduct](https://zio.dev/about/code-of-conduct)
## Support
Come chat with us on [![Badge-Discord]][Link-Discord].
[Badge-Discord]: https://img.shields.io/discord/629491597070827530?logo=discord "chat on discord"
[Link-Discord]: https://discord.gg/2ccFBr4 "Discord"
## License
[License](LICENSE)