{"id":27367156,"url":"https://github.com/durner/anyblob","last_synced_at":"2025-04-13T06:53:41.678Z","repository":{"id":177636300,"uuid":"644796312","full_name":"durner/AnyBlob","owner":"durner","description":"AnyBlob - A Universal Cloud Object Storage Download Manager Built For Cost-Throughput Optimal Analytics!","archived":false,"fork":false,"pushed_at":"2025-03-12T11:21:37.000Z","size":333,"stargazers_count":121,"open_issues_count":0,"forks_count":13,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-12T12:26:22.276Z","etag":null,"topics":["blob-storage","cloud","cloud-storage","downloader","object","object-storage","storage"],"latest_commit_sha":null,"homepage":"http://anyblob.com","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/durner.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-05-24T09:20:33.000Z","updated_at":"2025-03-12T11:21:33.000Z","dependencies_parsed_at":null,"dependency_job_id":"fb06c7ec-b31f-4fce-8a38-1b94c39e4b80","html_url":"https://github.com/durner/AnyBlob","commit_stats":null,"previous_names":["durner/anyblob"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/durner%2FAnyBlob","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/durner%2FAnyBlob/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/durner%2FAnyBlob/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/durner%2FAnyBlob/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/durner","download_url":"https://codeload.github.com/durner/AnyBlob/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248675454,"owners_count":21143767,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["blob-storage","cloud","cloud-storage","downloader","object","object-storage","storage"],"created_at":"2025-04-13T06:53:41.261Z","updated_at":"2025-04-13T06:53:41.670Z","avatar_url":"https://github.com/durner.png","language":"C++","readme":"# AnyBlob - Universal Cloud Object Storage Download Manager\n\nIn this repository, we present AnyBlob.\nAnyBlob is a universal download manager that allows to retrieve and upload objects to different cloud object stores.\nOur download manager uses less CPU resources than cloud-vendor provided libraries while retaining maximum throughput performance.\nAnyBlob leverages IO\\_uring for superior performance per core.\nFor experimental results, please visit our research paper at [PVLDB 16](https://www.vldb.org/pvldb/vol16/p2769-durner.pdf).\n\n## Building AnyBlob\n\nAnyBlob relies on some third-party libraries:\n- uring\n- openssl\n- jemalloc\n\nFor Ubuntu 22.04+ the following command installs the third-party libraries:\n```\nsudo apt update \u0026\u0026 sudo apt install liburing-dev openssl libssl-dev libjemalloc-dev lld g++ cmake\n```\n\nFor building, use the following commands:\n```\ngit clone --recursive https://github.com/durner/AnyBlob\nmkdir -p build/Release\ncd build/Release\ncmake -DCMAKE_BUILD_TYPE=Release ../..\nmake -j16\n```\n\n## Using AnyBlob\n\nAnyBlob is a modern C++ library that can be linked to your program.\nWe provide two examples how to use AnyBlob in your project.\nPlease find a simple example that does only download a single object from AWS, and a more sophisticated example that runs download benchmarks.\nThere, you can also find information on how to easily integrate the building process into existing CMake projects for an automated compilation and static linkage of AnyBlob.\nThe anyblob.cmake should be used in your project to integrate this repository as external project.\n\n## Contribution\n\nIf you have bug fixes or improvement, please do not hesitate to open a new merge request.\nFor coverage testing you can simply `make coverage` and open the coverage report found in the build directory at coverage/index.html.\n\n## Cite this work\n\nIf you are using AnyBlob in your scientific work, please cite:\n\n```\nExploiting Cloud Object Storage for High-Performance Analytics\nDominik Durner, Viktor Leis, and Thomas Neumann\nPVLDB 16 (11), 2023, 49th International Conference on Very Large Data Bases\n```\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdurner%2Fanyblob","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdurner%2Fanyblob","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdurner%2Fanyblob/lists"}