{"id":15291707,"url":"https://github.com/azure/blobporter","last_synced_at":"2025-04-15T16:38:28.322Z","repository":{"id":41183352,"uuid":"74929278","full_name":"Azure/blobporter","owner":"Azure","description":"Highly concurrent data transfer tool for Azure Blob Storage.","archived":false,"fork":false,"pushed_at":"2023-05-31T18:49:40.000Z","size":9253,"stargazers_count":85,"open_issues_count":12,"forks_count":29,"subscribers_count":20,"default_branch":"master","last_synced_at":"2025-04-09T10:11:57.745Z","etag":null,"topics":["azure-blob","azure-sdk","azure-storage","go"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Azure.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-11-28T02:22:44.000Z","updated_at":"2025-03-05T14:45:12.000Z","dependencies_parsed_at":"2024-06-20T15:42:14.723Z","dependency_job_id":"6544dd38-2e77-41bf-a5c2-c991b203f6a8","html_url":"https://github.com/Azure/blobporter","commit_stats":null,"previous_names":[],"tags_count":25,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Azure%2Fblobporter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Azure%2Fblobporter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Azure%2Fblobporter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Azure%2Fblobporter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Azure","download_url":"https://codeload.github.com/Azure/blobporter/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249109716,"owners_count":21214203,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["azure-blob","azure-sdk","azure-storage","go"],"created_at":"2024-09-30T16:14:01.286Z","updated_at":"2025-04-15T16:38:28.301Z","avatar_url":"https://github.com/Azure.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# BlobPorter\n\n[![Build Status](https://travis-ci.org/Azure/blobporter.svg?branch=master)](https://travis-ci.org/Azure/blobporter)\n[![Go Report Card](https://goreportcard.com/badge/github.com/Azure/blobporter)](https://goreportcard.com/report/github.com/Azure/blobporter)\n\n## Introduction\n\nBlobPorter is a data transfer tool for Azure Blob Storage that maximizes throughput through concurrent reads and writes that can scale up and down independently.\n\n![](img/bptransfer.png?raw=true)\n\nSources and targets are decoupled, this design enables the composition of various transfer scenarios.\n\n| From/To          | Azure Block Blob | Azure Page Blob  | File (Download) |\n| ---------------  | -----------------| -----------------|-----------------|\n| File (Upload)    | Yes              | Yes              | NA              |\n| HTTP/HTTPS*      | Yes              | Yes              | Yes             |\n| Azure Block Blob | Yes              | Yes              | Yes             |\n| Azure Page Blob  | Yes              | Yes              | Yes             |\n| S3 Endpoint      | Yes              | Yes              | No              |\n\n*\\*   The HTTP/HTTPS source must support HTTP byte ranges and return the file size as a response to a HTTP HEAD request.*\n\n### What's new (0.6.20) ?\n\n- Blob to Block Blobs transfers use the [Put Block from URL API](https://docs.microsoft.com/en-us/rest/api/storageservices/put-block-from-url).\n- Proxy support.\n- Minor fixes and performance improvements.\n- Upgrade to Go 1.11\n\n\n## Documentation\n\n\n* [Getting Started](http://blobporter.readthedocs.io/en/latest/gettingstarted.html)\n\n  * [Linux](http://blobporter.readthedocs.io/en/latest/gettingstarted.html#linux)\n\n  * [Windows](http://blobporter.readthedocs.io/en/latest/gettingstarted.html#windows)\n\n  * [Command Options](http://blobporter.readthedocs.io/en/latest/gettingstarted.html#command-options)\n\n* [Examples](http://blobporter.readthedocs.io/en/latest/examples.html)\n\n  * [Upload to Azure Block Blob Storage](http://blobporter.readthedocs.io/en/latest/examples.html#upload-to-azure-block-blob-storage)\n\n  * [Upload to Azure Page Blob Storage](http://blobporter.readthedocs.io/en/latest/examples.html#upload-to-azure-page-blob-storage)\n\n  * [Transfer data from S3 to Azure Storage](http://blobporter.readthedocs.io/en/latest/examples.html#transfer-data-from-s3-to-azure-storage)\n\n  * [Transfer data between Azure Storage accounts, containers and blob types](http://blobporter.readthedocs.io/en/latest/examples.html#transfer-data-between-azure-storage-accounts-containers-and-blob-types)\n\n  * [Transfer from an HTTP/HTTPS source to Azure Blob Storage](http://blobporter.readthedocs.io/en/latest/examples.html#transfer-from-an-http-https-source-to-azure-blob-storage)\n\n  * [Download from Azure Blob Storage](http://blobporter.readthedocs.io/en/latest/examples.html#download-from-azure-blob-storage)\n\n  * [Download a file from a HTTP source](http://blobporter.readthedocs.io/en/latest/examples.html#download-a-file-from-a-http-source)\n\n* [Resumable Transfers](http://blobporter.readthedocs.io/en/latest/resumabletrans.html)\n\n* [Performance Considerations](http://blobporter.readthedocs.io/en/latest/perfmode.html)\n\n    * [Best Practices](http://blobporter.readthedocs.io/en/latest/perfmode.html#best-practices)\n\n    * [Performance Measurement Mode](http://blobporter.readthedocs.io/en/latest/perfmode.html#performance-measurement-mode)\n\n## Issues and Feedback\n\nIf you have a question or find a bug, open a new issue in this repository. BlobPorter is an OSS project maintained by the contributors.\n\n## Contribute\n\nIf you would like to become an active contributor to this project please follow the instructions provided in [Microsoft Azure Projects Contribution Guidelines](http://azure.github.io/guidelines/).\n\n-----\nThis project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fazure%2Fblobporter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fazure%2Fblobporter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fazure%2Fblobporter/lists"}