{"id":22358653,"url":"https://github.com/deliganli/fstore4s","last_synced_at":"2025-06-13T23:36:59.336Z","repository":{"id":57718701,"uuid":"265555554","full_name":"Deliganli/fstore4s","owner":"Deliganli","description":"A common filestore interface for AWS S3, Google Cloud Storage and Local File Store","archived":false,"fork":false,"pushed_at":"2020-05-20T18:52:45.000Z","size":19,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-01-31T15:18:01.906Z","etag":null,"topics":["aws-s3","blob","blob-storage","blobstorage","file","file-storage","filestorage","fs2","gcs","google-cloud-storage","local-file"],"latest_commit_sha":null,"homepage":"","language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Deliganli.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-05-20T12:16:59.000Z","updated_at":"2022-09-17T15:38:16.000Z","dependencies_parsed_at":"2022-09-26T21:40:28.913Z","dependency_job_id":null,"html_url":"https://github.com/Deliganli/fstore4s","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Deliganli%2Ffstore4s","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Deliganli%2Ffstore4s/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Deliganli%2Ffstore4s/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Deliganli%2Ffstore4s/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Deliganli","download_url":"https://codeload.github.com/Deliganli/fstore4s/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245667744,"owners_count":20652983,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws-s3","blob","blob-storage","blobstorage","file","file-storage","filestorage","fs2","gcs","google-cloud-storage","local-file"],"created_at":"2024-12-04T15:16:40.912Z","updated_at":"2025-03-26T13:44:21.946Z","avatar_url":"https://github.com/Deliganli.png","language":"Scala","funding_links":[],"categories":[],"sub_categories":[],"readme":"# File Storage For Scala\n\nA common interface for doing basic file storage operations on multiple platforms\n\n## Install\n| Snapshot | Release |\n| --- | --- |\n| [![Sonatype Nexus (Releases)][Badge-SonatypeReleases]][Link-SonatypeReleases] | [![Sonatype Nexus (Snapshots)][Badge-SonatypeSnapshots]][Link-SonatypeSnapshots] |\n\n```sbt\n// Choose any you want to use\nlibraryDependencies += \"com.deliganli\" %% \"fstore4s-s3\"    % \"0.1.0\"\nlibraryDependencies += \"com.deliganli\" %% \"fstore4s-gcs\"   % \"0.1.0\"\nlibraryDependencies += \"com.deliganli\" %% \"fstore4s-local\" % \"0.1.0\"\n\n// You can add the core library if you want to implement your own file store\n// Other modules above use core inherently\nlibraryDependencies += \"com.deliganli\" %% \"fstore4s-core\" % \"0.1.0\"\n```\n## Details\n\nCurrent supported file storage platforms are;\n- Amazon Web Services (AWS) S3\n- Google Cloud Storage\n- Local file storage\n\nThe methods for manipulating data are same across all platforms.\n\n```scala\nstore.uri(address) // Generates signed uri valid over configured time\nstore.list(prefix) // Returns a stream with files under the prefix\n.\n.\n```\n\nPlatform can be changed easily by using another instance of `FileStorage[F,T]`\n\n```scala\nS3Store.create[IO, SimpleDomainFile](config, credentialProvider)\nGCSStore.create[IO, SimpleDomainFile](config)\nLocalFileStore.create[IO, SimpleDomainFile](config)\n```\n\nThose `create` methods are just helpers, one can create the `FileStore` directly\nby providing necessary underlying store instance\n\nUnifies storage providers' `InputStream`, `OutputStream`, `Iterator` and other java \nstreaming based functionality under `fs2.Stream`\n\n## What is Address?\n\nIt is basically a case class that has an implementation for `FileCodec`\n\n`FileCodec[T]` is used for naming the file from structured fields and vice versa\n\n`SimpleDomainFile` is a basic implementation that used across tests, simply it names \nthe stored files with 3 properties; version, category, name. For example;\n\n```scala\nAddress(\"test-dir\", SimpleDomainFile(\"V1\", \"attachment\", \"some-file.txt\"))\n```\n\nThis structured data can be used to encode the file to the filestore with the filename\nof `test-dir/V1_attachment_some-file.txt`. The `Address` class can be constructed \nfrom the filename as well so when we read filenames from the store directly we can \nhave our structured data back\n\nWith this way we can provide `case class -\u003e filename` and `filename -\u003e case class` \ntransformations that guarantees each unique case class corresponds to a unique file\n\n## This library is NOT\n\n- Sophisticated scala wrapper for particular file storage to use all possible functionality\n- Battle tested, production ready\n\n## This library might be useful at\n\n- Using one of the cloud file stores for production and using local store for tests\n- Quickly switch between file stores with minimal code change\n- Streaming applications that makes use of fs2\n\n## Bottom-Line\nThis library is very crude at the moment, not even sure there would be a demand since\nthis is an abstraction there will be the trade-offs between customizability and \nunification. If developed further, one probably will always find themselves deciding between\nproviding optional configuration that may not work with other vendors and not allowing\nany vendor specific usage\n\nThe drawbacks of 'melting everything in one pot' approach may discourage users with higher \nvolume and performance demands due to the lack of vendor-specific configuration. Though I \ntried to make everything streaming to make life easier for them as well.\n\n[Badge-SonatypeReleases]: https://img.shields.io/nexus/s/com.deliganli/fstore4s-core_2.13?server=https%3A%2F%2Foss.sonatype.org \"Sonatype Releases\"\n[Link-SonatypeReleases]: https://oss.sonatype.org/content/repositories/snapshots/com/deliganli/fstore4s-core_2.13/ \"Sonatype Releases\"\n\n[Badge-SonatypeSnapshots]: https://img.shields.io/nexus/r/https/oss.sonatype.org/com.deliganli/fstore4s-core_2.13.svg \"Sonatype Releases\"\n[Link-SonatypeSnapshots]: https://oss.sonatype.org/content/repositories/snapshots/com/deliganli/fstore4s-core_2.13/ \"Sonatype Releases\"\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeliganli%2Ffstore4s","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdeliganli%2Ffstore4s","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeliganli%2Ffstore4s/lists"}