{"id":19651281,"url":"https://github.com/embulk/embulk-output-azure_blob_storage","last_synced_at":"2025-07-19T10:09:52.145Z","repository":{"id":1755411,"uuid":"44046649","full_name":"embulk/embulk-output-azure_blob_storage","owner":"embulk","description":"Microsoft Azure Blob Storage file output plugin for Embulk","archived":false,"fork":false,"pushed_at":"2022-03-31T09:07:54.000Z","size":246,"stargazers_count":0,"open_issues_count":1,"forks_count":3,"subscribers_count":9,"default_branch":"master","last_synced_at":"2025-04-28T16:49:28.132Z","etag":null,"topics":["azure","azure-blob","embulk","embulk-output-plugin","embulk-plugin"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/embulk.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-10-11T09:42:09.000Z","updated_at":"2022-03-03T07:22:06.000Z","dependencies_parsed_at":"2022-08-06T11:01:09.817Z","dependency_job_id":null,"html_url":"https://github.com/embulk/embulk-output-azure_blob_storage","commit_stats":null,"previous_names":[],"tags_count":12,"template":false,"template_full_name":null,"purl":"pkg:github/embulk/embulk-output-azure_blob_storage","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/embulk%2Fembulk-output-azure_blob_storage","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/embulk%2Fembulk-output-azure_blob_storage/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/embulk%2Fembulk-output-azure_blob_storage/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/embulk%2Fembulk-output-azure_blob_storage/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/embulk","download_url":"https://codeload.github.com/embulk/embulk-output-azure_blob_storage/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/embulk%2Fembulk-output-azure_blob_storage/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265915154,"owners_count":23848491,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["azure","azure-blob","embulk","embulk-output-plugin","embulk-plugin"],"created_at":"2024-11-11T15:05:57.248Z","updated_at":"2025-07-19T10:09:52.114Z","avatar_url":"https://github.com/embulk.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Azure Blob Storage file output plugin for Embulk\n[![Build Status](https://travis-ci.org/embulk/embulk-output-azure_blob_storage.svg?branch=master)](https://travis-ci.org/embulk/embulk-output-azure_blob_storage)\n\n[Embulk](http://www.embulk.org/) file output plugin stores files on [Microsoft Azure](https://azure.microsoft.com/) [Blob Storage](https://azure.microsoft.com/en-us/documentation/articles/storage-introduction/#blob-storage)\n\n\n## Overview\n\n* **Plugin type**: file output\n* **Resume supported**: no\n* **Cleanup supported**: yes\n\n## Configuration\n\nFirst, create Azure [Storage Account](https://azure.microsoft.com/en-us/documentation/articles/storage-create-storage-account/).\n\n- **account_name**: storage account name (string, required)\n- **account_key**: primary access key (string, required)\n- **container**: container name (string, required)\n- **path_prefix**: prefix of target keys (string, required) (string, required)\n- **file_ext**: e.g. \"csv.gz, json.gz\" (string, required)\n- **blob_type**: `BLOCK_BLOB | PAGE_BLOB | APPEND_BLOB | UNSPECIFIED`, currently supports BLOCK_BLOB, other types will fall back into `UNSPECIFIED` (string, default `UNSPECIFIED`)\n\n\n### Auto create container\n\ncontainer will create automatically when container doesn't exists.\n \nWhen a container was deleted, a container with same name cannot be created for at least 30 seconds.\nIt's a [service specification](https://technet.microsoft.com/en-us/library/dd179408.aspx#Anchor_3) of Azure Blob Storage.\n\n## Example\n\n```yaml\nout:\n  type: azure_blob_storage\n  account_name: myaccount\n  account_key: myaccount_key\n  container: my-container\n  path_prefix: logs/csv-\n  file_ext: csv.gz\n  blob_type: BLOCK_BLOB\n  formatter:\n    type: csv\n    header_line: false\n  encoders:\n  - {type: gzip}\n```\n\n\n## Build\n\n```\n$ ./gradlew gem  # -t to watch change of files and rebuild continuously\n```\n\n\n## Test\n\n```\n$ ./gradlew test  # -t to watch change of files and rebuild continuously\n```\n\nTo run unit tests, we need to configure the following environment variables.\n\nWhen environment variables are not set, skip some test cases.\n\n```\nAZURE_ACCOUNT_NAME\nAZURE_ACCOUNT_KEY\nAZURE_CONTAINER\nAZURE_CONTAINER_DIRECTORY (optional, if needed)\n```\n\nIf you're using Mac OS X El Capitan and GUI Applications(IDE), like as follows.\n```xml\n$ vi ~/Library/LaunchAgents/environment.plist\n\u003c?xml version=\"1.0\" encoding=\"UTF-8\"?\u003e\n\u003c!DOCTYPE plist PUBLIC \"-//Apple//DTD PLIST 1.0//EN\" \"http://www.apple.com/DTDs/PropertyList-1.0.dtd\"\u003e\n\u003cplist version=\"1.0\"\u003e\n\u003cdict\u003e\n  \u003ckey\u003eLabel\u003c/key\u003e\n  \u003cstring\u003emy.startup\u003c/string\u003e\n  \u003ckey\u003eProgramArguments\u003c/key\u003e\n  \u003carray\u003e\n    \u003cstring\u003esh\u003c/string\u003e\n    \u003cstring\u003e-c\u003c/string\u003e\n    \u003cstring\u003e\n      launchctl setenv AZURE_ACCOUNT_NAME my-account-name\n      launchctl setenv AZURE_ACCOUNT_KEY my-account-key\n      launchctl setenv AZURE_CONTAINER my-container\n      launchctl setenv AZURE_CONTAINER_DIRECTORY unittests\n    \u003c/string\u003e\n  \u003c/array\u003e\n  \u003ckey\u003eRunAtLoad\u003c/key\u003e\n  \u003ctrue/\u003e\n\u003c/dict\u003e\n\u003c/plist\u003e\n\n$ launchctl load ~/Library/LaunchAgents/environment.plist\n$ launchctl getenv AZURE_ACCOUNT_NAME //try to get value.\n\nThen start your applications.\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fembulk%2Fembulk-output-azure_blob_storage","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fembulk%2Fembulk-output-azure_blob_storage","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fembulk%2Fembulk-output-azure_blob_storage/lists"}