Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

Projects in Awesome Lists tagged with s3

A curated list of projects in awesome lists tagged with s3 .

https://github.com/minio/minio

MinIO is a high-performance, S3 compatible object store, open sourced under GNU AGPLv3 license.

amazon-s3 cloud cloudnative cloudstorage go k8s kubernetes multi-cloud multi-cloud-kubernetes objectstorage s3 storage

Last synced: 16 Dec 2024

https://github.com/rclone/rclone

"rsync for cloud storage" - Google Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Wasabi, Google Cloud Storage, Azure Blob, Azure Files, Yandex Files

azure-blob azure-blob-storage azure-files backblaze-b2 cloud-storage dropbox encryption ftp fuse-filesystem go golang google-cloud-storage google-drive onedrive openstack-swift rclone s3 sftp sync webdav

Last synced: 16 Dec 2024

https://github.com/siyuan-note/siyuan

A privacy-first, self-hosted, fully open source personal knowledge management software, written in typescript and golang.

anki chatgpt electron evernote knowledge-base local-first markdown note-taking notebook notes-app notion obsidian ocr openai pdf pkm s3 self-hosted webdav

Last synced: 16 Dec 2024

https://github.com/seaweedfs/seaweedfs

SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, cross-DC active-active replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding.

blob-storage cloud-drive distributed-file-system distributed-storage distributed-systems erasure-coding fuse hadoop-hdfs hdfs kubernetes object-storage posix replication s3 s3-storage seaweedfs tiered-file-system

Last synced: 16 Dec 2024

https://github.com/airbytehq/airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.

bigquery change-data-capture data data-analysis data-collection data-engineering data-integration data-pipeline elt etl java mssql mysql pipeline postgresql python redshift s3 self-hosted snowflake

Last synced: 16 Dec 2024

https://github.com/thanos-io/thanos

Highly available Prometheus setup with long term storage capabilities. A CNCF Incubating project.

cncf go google-cloud-storage hacktoberfest high-availability metrics monitoring observability prometheus prometheus-ha-pairs prometheus-setup s3 storage thanos

Last synced: 16 Dec 2024

https://github.com/benbjohnson/litestream

Streaming replication for SQLite.

replication s3 sqlite

Last synced: 16 Dec 2024

https://github.com/juicedata/juicefs

JuiceFS is a distributed POSIX file system built on top of Redis and S3.

bigdata cloud-native distributed-systems filesystem go golang hdfs object-storage posix redis s3 storage

Last synced: 16 Dec 2024

https://github.com/mickael-kerjean/filestash

🦄 A file manager / web client for SFTP, S3, FTP, WebDAV, Git, Minio, LDAP, CalDAV, CardDAV, Mysql, Backblaze, ...

ftp ftps git ldap nfs s3 sftp smb webdav

Last synced: 16 Dec 2024

https://github.com/drakkan/sftpgo

Full-featured and highly configurable SFTP, HTTP/S, FTP/S and WebDAV server - S3, Google Cloud Storage, Azure Blob

azure-blob cloud-storage data-at-rest-encryption docker ftp ftp-server git go google-cloud-storage multi-factor-authentication portable prometheus proxy-protocol s3 scp sftp sftp-server webdav webdav-server

Last synced: 16 Dec 2024

https://github.com/s3fs-fuse/s3fs-fuse

FUSE-based file system backed by Amazon S3

aws-s3 cloud-storage filesystem fuse fuse-filesystem s3

Last synced: 25 Oct 2024

https://github.com/getmoto/moto

A library that allows you to easily mock out tests based on AWS infrastructure.

aws boto ec2 s3

Last synced: 16 Dec 2024

https://github.com/spulec/moto

A library that allows you to easily mock out tests based on AWS infrastructure.

aws boto ec2 s3

Last synced: 02 Nov 2024

https://github.com/backup/backup

Easy full stack backup operations on UNIX-like systems.

backup database encryption mongodb mysql postgresql ruby s3 slack

Last synced: 17 Dec 2024

https://github.com/meskyanichi/backup

Easy full stack backup operations on UNIX-like systems.

backup database encryption mongodb mysql postgresql ruby s3 slack

Last synced: 05 Dec 2024

https://github.com/s3tools/s3cmd

Official s3cmd repo -- Command line tool for managing S3 compatible storage services (including Amazon S3 and CloudFront).

backup cli hacktoberfest python s3 s3cmd storage-service

Last synced: 17 Dec 2024

https://github.com/awslabs/mountpoint-s3

A simple, high-throughput file client for mounting an Amazon S3 bucket as a local file system.

aws filesystem fuse s3

Last synced: 16 Dec 2024

https://github.com/qovery/replibyte

Seed your development database with real data ⚡️

aws backup cloud cloudnative database mongodb mysql postgres postgresql rust rust-lang s3

Last synced: 17 Dec 2024

https://github.com/Qovery/Replibyte

Seed your development database with real data ⚡️

aws backup cloud cloudnative database mongodb mysql postgres postgresql rust rust-lang s3

Last synced: 25 Oct 2024

https://github.com/automq/automq

AutoMQ is a cloud-first alternative to Kafka by decoupling durability to S3 and EBS. 10x Cost-Effective. No Cross-AZ Traffic Cost. Autoscale in seconds. Single-digit ms latency.

aws azure cloud cloud-economics cloud-first cloud-native ebs gcp kafka messaging minio s3 serverless spot storage streaming

Last synced: 17 Dec 2024

https://github.com/AutoMQ/automq

AutoMQ is a cloud-first alternative to Kafka by decoupling durability to S3 and EBS. 10x Cost-Effective. No Cross-AZ Traffic Cost. Autoscale in seconds. Single-digit ms latency.

aws azure cloud cloud-economics cloud-first cloud-native ebs gcp kafka messaging minio s3 serverless spot storage streaming

Last synced: 01 Nov 2024

https://github.com/iterate-ch/cyberduck

Cyberduck is a libre FTP, SFTP, WebDAV, Amazon S3, Backblaze B2, Microsoft Azure & OneDrive and OpenStack Swift file transfer client for Mac and Windows.

azure backblaze-b2 cloudfront cryptomator cyberduck dracoon dropbox ftp google-drive google-storage hubic irods onedrive onedrive-for-business onedrive-storage openstack-swift s3 sftp sharepoint webdav

Last synced: 18 Dec 2024

https://github.com/apache/opendal

Apache OpenDAL: access data freely.

azblob gcs hacktoberfest redis rust s3 storage

Last synced: 16 Dec 2024

https://github.com/roapi/roapi

Create full-fledged APIs for slowly moving datasets without writing a single line of code.

analytics arrow blob-storage cloud-native columnar datafusion datasets delta-lake graphql in-memory-database parquet query query-frontends rest-api rust s3 sql static-datasets

Last synced: 30 Oct 2024

https://github.com/piskvorky/smart_open

Utils for streaming large files (S3, HDFS, gzip, bz2...)

boto bz2 file gzip-stream hacktoberfest hdfs python s3 streaming streaming-data webhdfs

Last synced: 16 Dec 2024

https://github.com/shrinerb/shrine

File Attachment toolkit for Ruby applications

attachment background-jobs direct-upload file-upload filesystem metadata orm rack ruby s3 storage

Last synced: 16 Dec 2024

https://github.com/aws/aws-sdk-js-v3

Modularized AWS SDK for JavaScript.

aws-sdk aws-sdk-js dynamodb javascript lambda nodejs s3 sns typescript

Last synced: 16 Dec 2024

https://github.com/minio/mc

Unix like utilities for object store

aws aws-s3 coreutils filesystem go golang s3 s3-bucket storage tools

Last synced: 16 Dec 2024

https://github.com/winscp/winscp

WinSCP is a popular free file manager for Windows supporting SFTP, FTP, FTPS, SCP, S3, WebDAV and local-to-local file transfers. A powerful tool to enhance your productivity with a user-friendly interface and automation options like .NET assembly.

ftp ftp-client ftps s3 scp sftp ssh webdav windows winscp

Last synced: 19 Dec 2024

https://github.com/peak/s5cmd

Parallel S3 and local filesystem execution tool.

aws cli filesystem go s3 s5cmd storage

Last synced: 16 Dec 2024

https://github.com/sa7mon/s3scanner

Scan for misconfigured S3 buckets across S3-compatible APIs!

aws bugbounty gcp infosec s3 s3scanner

Last synced: 17 Dec 2024

https://github.com/sa7mon/S3Scanner

Scan for misconfigured S3 buckets across S3-compatible APIs!

aws bugbounty gcp infosec s3 s3scanner

Last synced: 28 Oct 2024

https://github.com/ladjs/lad

Node.js framework made by a former @expressjs TC and @koajs team member. Built for @forwardemail, @spamscanner, @breejs, @cabinjs, and @lassjs.

api async-await ava aws boilerplate bootstrap cloudfront framework javascript koa mongodb mongoose mvc nodejs passport prettier react-native redis s3 xo

Last synced: 17 Dec 2024

https://github.com/minio/minio-go

MinIO Go client SDK for S3 compatible object storage

aws-signature client-sdk cloud fileuploader go libraries minio s3

Last synced: 16 Dec 2024

https://github.com/minio/minio-Go

MinIO Go client SDK for S3 compatible object storage

aws-signature client-sdk cloud fileuploader go libraries minio s3

Last synced: 24 Oct 2024

https://github.com/peerdb-io/peerdb

Fast, Simple and a cost effective tool to replicate data from Postgres to Data Warehouses, Queues and Storage

bigquery cdc clickhouse cloud-native distributed-systems etl eventhubs kafka postgres postgresql realtime rust s3 snowflake sql stream-processing

Last synced: 19 Dec 2024

https://github.com/byconity/byconity

ByConity is an open source cloud data warehouse

bytedance clickhouse clickhouse-database cloud kubernets lakehouse olap s3 snowflake sql tiktok

Last synced: 19 Dec 2024

https://github.com/PeerDB-io/peerdb

Fast, Simple and a cost effective tool to replicate data from Postgres to Data Warehouses, Queues and Storage

bigquery cdc clickhouse cloud-native distributed-systems etl eventhubs kafka postgres postgresql realtime rust s3 snowflake sql stream-processing

Last synced: 31 Oct 2024

https://github.com/ByConity/ByConity

ByConity is an open source cloud data warehouse

bytedance clickhouse clickhouse-database cloud kubernets lakehouse olap s3 snowflake sql tiktok

Last synced: 27 Oct 2024

https://github.com/QingWei-Li/notea

📒 Self hosted note taking app stored on S3

docker markdown minio note-taking s3

Last synced: 15 Dec 2024

https://github.com/notea-org/notea

📒 Self hosted note taking app stored on S3

docker markdown minio note-taking s3

Last synced: 13 Dec 2024

https://github.com/offen/docker-volume-backup

Backup Docker volumes locally or to any S3, WebDAV, Azure Blob Storage, Dropbox or SSH compatible storage

backup docker docker-compose docker-swarm docker-volume s3 ssh webdav

Last synced: 19 Dec 2024

https://github.com/jstarfish/javakeeper

✍️ Java 工程师必备架构体系知识总结:涵盖分布式、微服务、RPC等互联网公司常用架构,以及数据存储、缓存、搜索等必备技能

algorithms elasticsearch java jvm kafka linux mysql redis s3 zookeeper

Last synced: 19 Dec 2024

https://github.com/humanmade/s3-uploads

The WordPress Plugin to Store Uploads on Amazon S3

s3 wordpress wordpress-plugin

Last synced: 16 Dec 2024

https://github.com/humanmade/S3-Uploads

The WordPress Plugin to Store Uploads on Amazon S3

s3 wordpress wordpress-plugin

Last synced: 02 Nov 2024

https://github.com/Jstarfish/JavaKeeper

✍️ Java 工程师必备架构体系知识总结:涵盖分布式、微服务、RPC等互联网公司常用架构,以及数据存储、缓存、搜索等必备技能

algorithms elasticsearch java jvm kafka linux mysql redis s3 zookeeper

Last synced: 29 Oct 2024

https://github.com/saltbo/zpan

A self-hosted cloud disk base on the cloud storage./ 一个基于云存储的网盘系统,用于自建私人网盘或企业网盘。

cloud cloud-disk cloud-storage disk oss qiniuyun s3 upyun

Last synced: 20 Dec 2024

https://github.com/gaul/s3proxy

Access other storage backends via the S3 API

atmos aws-s3 azure backblaze-b2 google-cloud-storage openstack-swift proxy s3

Last synced: 17 Dec 2024

https://github.com/dromara/x-file-storage

一行代码将文件存储到 本地、FTP、SFTP、WebDAV、谷歌云存储、阿里云OSS、华为云OBS、七牛云Kodo、腾讯云COS、百度云 BOS、又拍云USS、MinIO、 AWS S3、FastDFS、 Azure Blob Storage、金山云 KS3、美团云 MSS、京东云 OSS、天翼云 OOS、移动云 EOS、沃云 OSS、 网易数帆 NOS、Ucloud US3、青云 QingStor、平安云 OBS、首云 OSS、IBM COS、其它兼容 S3 协议的平台。后续即将支持 Samba、NFS

bos file-storage file-upload ftp java kodo minio oss s3 sftp spring uss webdav

Last synced: 19 Dec 2024

https://github.com/rilldata/rill

Rill is a tool for effortlessly transforming data sets into powerful, opinionated dashboards using SQL. BI-as-code.

bi business-analytics csv data data-analysis data-visualization dataviz duckdb gcs golang parquet parquet-tools parquet-viewer s3 sql sql-editor svelte sveltejs sveltekit

Last synced: 17 Dec 2024

https://github.com/cloudkj/scar

Deploy static websites in seconds - with HTTPS, a global CDN, and custom domains.

aws cloudformation cloudfront route53 s3 serverless static-site

Last synced: 15 Dec 2024

https://github.com/aws/aws-toolkit-vscode

Amazon Q, CodeCatalyst, Local Lambda debug, SAM/CFN syntax, ECS Terminal, AWS resources

amazon amazon-q aws cdk cloudformation codecatalyst codewhisperer lambda s3 serverless vscode vscode-extension

Last synced: 18 Dec 2024

https://github.com/AlexAkulov/clickhouse-backup

Tool for easy backup and restore for ClickHouse® using object storage for backup files.

azblob backup clickhouse clickhouse-backup clickhousedump dump ftp gcs s3 sftp

Last synced: 11 Dec 2024

https://github.com/altinity/clickhouse-backup

Tool for easy backup and restore for ClickHouse® using object storage for backup files.

azblob backup clickhouse clickhouse-backup clickhousedump dump ftp gcs s3 sftp

Last synced: 17 Dec 2024

https://github.com/Altinity/clickhouse-backup

Tool for easy backup and restore for ClickHouse® using object storage for backup files.

azblob backup clickhouse clickhouse-backup clickhousedump dump ftp gcs s3 sftp

Last synced: 05 Nov 2024

https://github.com/swoodford/aws

A collection of bash shell scripts for automating various tasks with Amazon Web Services using the AWS CLI and jq.

amazon aws aws-cli awscli bash cidr cloudfront cloudwatch ec2 elb iam infrastructure-as-code jq json route53 s3 shell terraform vpc waf

Last synced: 19 Dec 2024

https://github.com/nhost/hasura-backend-plus

🔑Auth and 📦Storage for Hasura. The quickest way to get Auth and Storage working for your next app based on Hasura.

authentication docker graphql hasura jwt s3 storage typescript

Last synced: 21 Dec 2024

https://github.com/stavro/arc

:paperclip: Flexible file upload and attachment library for Elixir

attachment elixir s3 s3-storage storage

Last synced: 17 Dec 2024

https://github.com/littlstar/s3-lambda

Lambda functions over S3 objects with concurrency control (each, map, reduce, filter)

concurrency lambda-functions node s3

Last synced: 16 Dec 2024

https://github.com/algotraders/stock-analysis-engine

Backtest 1000s of minute-by-minute trading algorithms for training AI with automated pricing data from: IEX, Tradier and FinViz. Datasets and trading performance automatically published to S3 for building AI training datasets for teaching DNNs how to trade. Runs on Kubernetes and docker-compose. >150 million trading history rows generated from +5000 algorithms. Heads up: Yahoo's Finance API was disabled on 2019-01-03 https://developer.yahoo.com/yql/

algorithmic-trading backtesting deep-learning deep-learning-tutorial deep-neural-networks docker helm helm-charts iex iexcloud jupyter keras kubernetes minio options redis s3 stocks tensorflow tradier

Last synced: 21 Dec 2024

https://github.com/jakejarvis/s3-sync-action

🔄 GitHub Action to sync a directory with a remote S3 bucket 🧺

actions aws aws-s3 ci deployment github-actions s3

Last synced: 20 Dec 2024

https://github.com/AlgoTraders/stock-analysis-engine

Backtest 1000s of minute-by-minute trading algorithms for training AI with automated pricing data from: IEX, Tradier and FinViz. Datasets and trading performance automatically published to S3 for building AI training datasets for teaching DNNs how to trade. Runs on Kubernetes and docker-compose. >150 million trading history rows generated from +5000 algorithms. Heads up: Yahoo's Finance API was disabled on 2019-01-03 https://developer.yahoo.com/yql/

algorithmic-trading backtesting deep-learning deep-learning-tutorial deep-neural-networks docker helm helm-charts iex iexcloud jupyter keras kubernetes minio options redis s3 stocks tensorflow tradier

Last synced: 01 Nov 2024

https://github.com/lensesio/stream-reactor

A collection of open source Apache 2.0 Kafka Connector maintained by Lenses.io.

awss3 azure-datalake cassandra connector cosmosdb elasticsearch eventhubs gcp http influxdb jms kafka kafka-connect mongodb mqtt pubsub s3

Last synced: 19 Dec 2024

https://github.com/bareos/bareos

Bareos is a cross-network Open Source backup solution (licensed under AGPLv3) which preserves, archives, and recovers data from all major operating systems.

archiving backup backup-solution backup-utility bareos ceph compression cross-platform disaster-recovery encrypt gluster mysql postgresql python recover restore s3 security vmware

Last synced: 17 Dec 2024

https://github.com/SnellerInc/sneller

World's fastest log analysis: λ + SQL + JSON + S3

avx512 go high-performance indexless json log query-engine s3 schemaless serverless simd sql vectorized

Last synced: 28 Oct 2024

https://github.com/ysde/grafana-backup-tool

A Python-based application to backup Grafana settings by using the Grafana API

backup docker gcs grafana s3

Last synced: 17 Dec 2024

https://github.com/apache/ozone

Scalable, redundant, and distributed object store for Apache Hadoop

big-data hadoop kubernetes object-store s3 storage

Last synced: 19 Dec 2024

https://github.com/adobe/S3Mock

A simple mock implementation of the AWS S3 API startable as Docker image, TestContainer, JUnit 4 rule, JUnit Jupiter extension or TestNG listener

amd64 arm64 aws aws-s3 docker java junit kotlin mock s3 testcontainers testing

Last synced: 06 Nov 2024

https://github.com/adobe/s3mock

A simple mock implementation of the AWS S3 API startable as Docker image, TestContainer, JUnit 4 rule, JUnit Jupiter extension or TestNG listener

amd64 arm64 aws aws-s3 docker java junit kotlin mock s3 testcontainers testing

Last synced: 17 Dec 2024

https://github.com/tmspzz/Rome

Carthage cache for S3, Minio, Ceph, Google Storage, Artifactory and many others

aws aws-s3 cache carthage ceph cocoapods fastlane fastlane-plugin framework hacktoberfest ios macos minio package-manager s3 swift tool tvos watchos

Last synced: 07 Dec 2024

https://github.com/dvassallo/s3-benchmark

Measure Amazon S3's performance from any location.

aws benchmark performance-analysis s3

Last synced: 27 Oct 2024

https://github.com/sambecker/exif-photo-blog

Photo blog, reporting 🤓 EXIF camera details (aperture, shutter speed, ISO) for each image.

ai approuter blob-storage exif nextjs ogimage photography postgresql r2 s3 vercel

Last synced: 20 Dec 2024

https://github.com/supabase/storage

S3 compatible object storage service that stores metadata in Postgres

gotrue object-storage postgres s3

Last synced: 20 Dec 2024

https://github.com/ctrox/csi-s3

A Container Storage Interface for S3

csi golang k8s kubernetes s3

Last synced: 31 Oct 2024

https://github.com/amazon-archives/aws-mobile-react-sample

A React Starter App that displays how web developers can integrate their front end with AWS on the backend. The App interacts with AWS Cognito, API Gateway, Lambda and DynamoDB on the backend.

api-gateway aws aws-amplify aws-amplify-react cognito dynamodb javascript lambda mobile-hub react s3 serverless

Last synced: 08 Nov 2024

https://github.com/bradleyg/django-s3direct

Directly upload files to S3 compatible services with Django.

aws django django-s3direct python s3 storages uploads

Last synced: 29 Oct 2024

https://github.com/brandonweiss/discharge

⚡️ A simple, easy way to deploy static websites to Amazon S3.

aws aws-s3 deployment s3 static-site

Last synced: 25 Oct 2024

https://github.com/deuxfleurs-org/garage

(Mirror) S3-compatible object store for small self-hosted geo-distributed deployments. Main repo: https://git.deuxfleurs.fr/Deuxfleurs/garage

object-storage rust s3

Last synced: 09 Nov 2024

https://github.com/ryanto/next-s3-upload

Upload files from your Next.js app to S3

nextjs s3

Last synced: 19 Dec 2024