{"id":13825059,"url":"https://github.com/tarwn/BasicAzureStorageSDK","last_synced_at":"2025-07-08T21:31:00.498Z","repository":{"id":13948060,"uuid":"16648074","full_name":"tarwn/BasicAzureStorageSDK","owner":"tarwn","description":"A readable, testable, and more straightforward .Net SDK for the Azure Storage API","archived":true,"fork":false,"pushed_at":"2018-02-17T18:59:57.000Z","size":572,"stargazers_count":9,"open_issues_count":0,"forks_count":4,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-11-20T03:32:01.366Z","etag":null,"topics":["azure","csharp"],"latest_commit_sha":null,"homepage":"","language":"C#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tarwn.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2014-02-08T17:08:42.000Z","updated_at":"2023-01-28T13:55:53.000Z","dependencies_parsed_at":"2022-08-24T21:41:35.804Z","dependency_job_id":null,"html_url":"https://github.com/tarwn/BasicAzureStorageSDK","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/tarwn/BasicAzureStorageSDK","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tarwn%2FBasicAzureStorageSDK","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tarwn%2FBasicAzureStorageSDK/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tarwn%2FBasicAzureStorageSDK/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tarwn%2FBasicAzureStorageSDK/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tarwn","download_url":"https://codeload.github.com/tarwn/BasicAzureStorageSDK/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tarwn%2FBasicAzureStorageSDK/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":264352624,"owners_count":23594931,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["azure","csharp"],"created_at":"2024-08-04T09:01:14.179Z","updated_at":"2025-07-08T21:31:00.151Z","avatar_url":"https://github.com/tarwn.png","language":"C#","funding_links":[],"categories":["C# #"],"sub_categories":[],"readme":"BasicAzureStorageSDK\n====================\n\nThis is a .Net SDK designed to closely match the Azure Storage API, focused mostly on Queue and Blob storage \n(with limited support for Table). The operations align with the API documentation, it supports only features \nthat can be tested locally against the storage emulator, and provides interfaces and no internal methods.\n\n* .Net Framework Version: 4.5 or newer\n* Azure Storage API Version: 2013-08-15\n* NuGet Dependencies: Netwonsoft.Json, TransientFaultHandling.Core\n\nSupported Operations:\n\n* Queue Service - 15/17 (missing preflight + service stats)\n* Blob Service - 20/31 (missing account-level calls, page, append-only)\n* Table Service - 7 + 2 halves/16\n* File Service - 0/19\n\nThe library includes:\n\n* Interfaces or overridable methods, for easier unit testing\n* Built-in Retry Policies\n* Strongly-typed Exceptions for all service errors, ex: BlobNotFoundAzureException\n* Service clients (blob, queue, table) that closely match the Storage API Documentation\n* Helper methods for automatically managing large block blobs\n* Sync and Async implementations of every method\n* Unit and Integration tests for every service method (verified against the Storage SDK)\n\nLicense\n====================\n\nSee LICENSE.txt\n\nGetting Started\n====================\n\nThere are 2 key ingredients in every call to Azure Storage, the StorageAccountSettings and a ServiceClient.\n\nHere's a demo of creating a queue, adding an item to it, and reading that item:\n\n\tvar accountSettings = StorageAccountSettings.Parse(\"UseDevelopmentStorage=true\");\n\tvar client = new QueueServiceClient(accountSettings);\n\n\tclient.CreateQueue(\"myawesomequeue\");\n\n\tclient.PutMessage(\"myawesomequeue\", \"my test message!\");\n\n\tvar response = client.GetMessages(\"myawesomequeue\", 1);\n\n\t//do something with response.Messages[0]\n\n\tclient.DeleteMessage(\"myawesomequeue\", response.Messages[0].Id, response.Messages[0].PopReceipt);\n\nAll methods have both a synchronous and Async version, so this can also be written like so:\n\n\tvar accountSettings = StorageAccountSettings.Parse(\"UseDevelopmentStorage=true\");\n\tvar client = new QueueServiceClient(accountSettings);\n\n\tawait client.CreateQueueAsync(\"myawesomequeue\");\n\n\tawait client.PutMessageAsync(\"myawesomequeue\", \"my test message!\");\n\n\tvar response = await client.GetMessagesAsync(\"myawesomequeue\", 1);\n\n\t//do something with response.Messages[0]\n\n\tawait client.DeleteMessageAsync(\"myawesomequeue\", response.Messages[0].Id, response.Messages[0].PopReceipt);\n\nThe methods in the QueueServiceClient match the [API documentation for the Queue service](http://msdn.microsoft.com/en-us/library/azure/dd179363.aspx) closely.\n\n## StorageAccountSettings\n\nYou can define the storage account settings with either the constructor or by feeding in an [Azure Storage \nconnection string](https://azure.microsoft.com/en-us/documentation/articles/storage-configure-connection-string/) like you would the official SDK:\n\n\t// some examples with connection strings\n\tvar accountSettings = StorageAccountSettings.Parse(\"UseDevelopmentStorage=true\");\n\tvar accountSettings = StorageAccountSettings.Parse(\"AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;\");\n\tvar accountSettings = StorageAccountSettings.Parse(\"AccountName=some-account-name;AccountKey=some-account-key;\");\n\n\t// or the constructor\n\tvar accountSettings = new StorageAccountSettings(\"devstoreaccount1\", \"Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==\");\n\tvar accountSettings = new StorageAccountSettings(\"some-account-name\", \"some-account-key\");\n\nSupport is included for changing HTTP/HTTPS, providing alternate blob/table/queue endpoints, and address suffixes \nother than \"core.windows.net\". \n\n(See [StorageAccountSettingsTests](https://github.com/tarwn/BasicAzureStorageSDK/blob/master/Basic.Azure.Storage.Tests/StorageAccountSettingsTests.cs) for test examples of the different connection strings that can be parsed).\n\n\nUsing the library\n====================\n\nThere is not a published NuGet package for this library (coming soon).\n\nTo add this to a project:\n\n* Clone the Download [NuGet.exe](https://dist.nuget.org/index.html)\n* Build the project in Release mode\n* Run `nuget.exe pack`\n* Copy the resulting nuget package to a folder in your solution and add that folder as a nuget repository\n\nUpgrading and Additions\n========================\n\nFeel free to make upgrades and additions.\n\n1. Create a Github issue so we can talk ahead of time (it may not require much, or there\nmay be somethign non-obvious I can point out)\n\n2. Make the change and send a PR for review: As long as the commit messages are descriptive\nand include the issue #, we're good, this isnt' a highly trafficked library.\n\nUpgrading workflow\n----------------------\n\nTo upgrade the version of the storage API:\n\n1. Locate the version string in the MSDN docs\n\n2. Update the value in Basic/Azure.Storage/Communications/Core/RequestBase, ln 17\n\n3. Run all the tests, make fixes as necessary to support changes in the API (stick to their terminology and structure as much as possible)\n\n4. Update the 4 error TT files in ServceExceptions and regenerate them (instructions present in each file, should be mostly copy/paste/replace/paste some more).\n\nGeography\n====================\n\nThe library tries to stick as close to the API structure and naming as possible.\n\nUsing the Library\n--------------------\n\nThese are the objects you would access while using the library.\n\n### Clients\n\n- BlobStorageClient - operations for Blob Storage Service - http://msdn.microsoft.com/en-us/library/azure/dd135733.aspx\n- QueueServiceClient - operations for Queue Service - http://msdn.microsoft.com/en-us/library/azure/dd179363.aspx\n- TableServiceClient - operations for Table Service - http://msdn.microsoft.com/en-us/library/azure/dd179423.aspx\n\nOperations on each of these clients are named the same as they are in the corresponding API documentation.\nParameters that are required in the documentation are required by operations in the client, parameters that\nare optional in the documentation are optional in the client. If there is only a limited set of input options\nallowed, an enum will exist that has those options.\n\n### Differences\n\nFor the most part, each API call will have 1 matching Request object. However, there are a few API calls that\nare actually multiple actions in a single call, and for those I have decided to make alter the pattern so the\nend consumer only has to deal with what is required/allowed for a specific action of the call rather than\nall possible inputs for all possible actions on one request.\n\n**PutBlob** has been split into a PutBlockBlob and PutPageBlob. In the API, some of the parameters are truly\noptional and some are only optional depending on which flavor you are uploading, so I chose to split it into\ntwo methods so that only the truly optional parameters would be optional and the ones that are type dependeant\nare not available for the opposite call.\n\n**LeaseBlob** Has 5 actions with different allowed and required inputs and different expected responses.\nI have split this into 5 calls because I would prefer to make it clear and simple which arguments are\nnecessary for each call, even if it does somewhat break the model of 1 API call = 1 Request object.\n\n**LeaseContainer** Has 5 actions with different allowed and required inputs and different expected responses.\nI have split this into 5 calls because I would prefer to make it clear and simple which arguments are\nnecessary for each call, even if it does somewhat break the model of 1 API call = 1 Request object.\n\nNote: This is an area that I find questionable in the API. I'm not sure why they chose to embed multiple\nactions into common requests like this, I think they generally did a good idea of making the API clear and\nthese points stand out in contradiction to the rest.\n\n### Responses\n\nStorage API calls that returns responses will have a matching response object defined in the library that\nis named after the operation. Executing a CreateContainer operation, for instance, will return a\nCreateContainerResponse.\n\n### Exceptions\n\nThere is a specific exception for each error listed in the documented API error tables.\n\n- Common Service Errors\n- Blob Service Errors\n- Queue Service Errors\n- Table Service Errors\n\nReference: http://msdn.microsoft.com/en-us/library/azure/dd179382.aspx\n\nThey all share a common base AzureException, so you have the option of handling all Azure Exceptions in a single\ncatch statement, or add logic for specific errors (such as BlobNotFoundAzureException).\n\n### Retries\n\nThere is a default retry Policy currently in place that will be exposed later. When this retry policy is exhausted,\nit wraps the final specific exception in a RetriedException so you have both the final exception details as well\nas information on how many times the operation was tried before giving up.\n\nInternal Geography\n-------------------\n\nThe layout of the code internally follows some patterns as well.\n\n- /ClientContracts - the interfaces for the Blob/Table/Queue client objects above\n- /Communications/BlobService - the blob service operations\n- /Communications/Common - common data objects and enums used in API operations\n- /Communications/Core - the core logic for Requests and Responses distilled into one place so Request/Response for Operations only reflect the specific requirements for the operation\n- /Communications/QueueService - the Queue Service operations\n- /Communications/ServiceExceptions - concrete exception for each documented service exception, a base AzureException, and an AzureResponseParseException\n- /Communications/TableService - the Table Service operations\n- /Communications/Utility - utility classes for parsing and formatting of data used by Request and Response objects\n\n### Service Folder Structures\n\nThe services folder structure follows the documentation folder structure, with a single high level folder for the\nservice, sub-folders for the categories of operations, and then Request and Response objects named the same as the\nAPI operations.\n\nExample: Create Container - http://msdn.microsoft.com/en-us/library/windowsazure/dd179468.aspx\nLibrary Location: /Communications/BlobService/ContainerOperations/CreateContainerRequest\nDocumentation: Blob Service / Operations on Containers / Create Contaner\n\n_Note: I'm not sure if MSDN renamed the \"Operations\" level or if I was inconsistent about it, the documentation has\nmoved since I initially started this._\n\n### Service Exceptions\n\nService Exceptions are generated from the documentated tables of errors in the API, using regular expressions to\ntweak the output and then T4 to generate the actual classes. They have a common base class, giving you the freedom\nto catch exceptions as specifically or generally as you want.\n\nThese exceptions also include all of the details available. The API often provides more details for errors than are\nsurfaced in the standard Azure Storage SDK; these exceptions surface all of that additional information.\n\n### Retried Exceptions\n\nWhen the retry policy is exhausted, the final exception is wrapped in a RetriedException to provide information that\nthe Operation was retried multiple times, including the number of times it was tried. This is often one of the first\nthings that Azure support asks. later I may also add the full collection of exceptions that happened, as there is at\nleast one outstanding issue around popreceipts that requires you to know about the first error that occurred.\n\n### Core Logic\n\nThe core logic includes the RequestBase, the Response wrapper that wraps around the specific resposne payload\nexpected for an operation, the RetriedException that is returned when we give up after exhausting the retry policy,\nand various constants for header values and such, the logic to create signed authorization headers, etc.\n\nImplemented Methods\n====================\n\nThis section will list all of the available methods from the documentation, as of 2014-10-13, and whether they have\nbeen implemented yet.\n\n- Queue is mostly done - 15/17 operations\n- Table is started - 7 + 2 halves/16 operations\n- Blob is pretty far - 20/31 operations\n- File is not present at all\n\nQueue Service - 15/17 - QueueServiceClient: IQueueServiceClient\n-----------------------------------------------------------\n\nAccount Operations\n\n- List Queues - Complete - does not automatically download with continuation markers\n- Set Queue Service Properties - Complete - does not include CORS additions from 2013\n- Get Queue Servuce Properties - Complete - does not include CORS additions from 2013\n- Preflight Queue Request - No\n- Get Queue Service States - No\n\nQueue Operations\n\n- Create Queue - Yes\n- Delete Queue - Yes\n- Get Queue Metadata - Yes\n- Set Queue Metadata - Yes\n- Get Queue ACL - Yes\n- Set Queue ACL - Yes\n\nMessage Operations\n\n- Put Message - Yes\n- Get Messages - Yes\n- Peek Messages - Yes\n- Delete Message - Yes\n- Clear Messages - Yes - Does not auto-retry the 500 Operation Timeout yet\n- Update Message - Yes\n\nBlob Service - 20/31 - BlobServiceClient: IBlobServiceClient\n-----------------------------------------------------------\n\nAccount Operations\n\n- List Containers - No\n- Set Blob Service Properties - No\n- Get Blob Service Properties - No\n- Preflight Blob Request - No\n- Get Blob Service States - No\n\nContainer Operations\n\n- Create Container - Yes\n- Get Container Properties - Yes\n- Get Container Metadata - Yes\n- Set Container Metadata - Yes\n- Get Container ACL - Yes\n- Set Container ACL - Yes\n- Delete Container - Yes\n- Lease Container - Yes\n- List Blobs - Yes\n\nBlob Operations\n\n- Put Blob - Yes - two flavors (see notes above): PutBlockBlob and PutPageBlob\n- Get Blob - Yes\n- Get Blob Properties - Yes\n- Set Blob Properties - No\n- Get Blob Metadata - Yes\n- Set Blob Metadata - Yes\n- Lease Blob - Yes\n- Snapshot Blob - No\n- Copy Blob - Yes\n- Abort Copy Blob - No\n- Delete Blob - Yes\n\nBlock Blob Operations\n\n- Put Block - Yes\n- Put Block List - Yes\n- Get Block List - Yes\n\nPage Blob Operations\n\n- Put Page - No\n- Get Page Ranges - No\n\nAppend Blob Operations\n\n- Append Block - No\n\n\nTable Service - 7 + 2 halves/16 - TableServiceClient: ITableServiceClient\n-----------------------------------------------------------\n\nAccount Operations\n\n- Set Table Service Properties - No\n- Get Table Service Properties - No\n- Preflight Table Request - No\n- Get Table Service Stats - No\n\nTable Operations\n\n- Query Tables - Basic (no OData support yet)\n- Create Table - Yes\n- Delete Table - No\n- Get Table ACL - No\n- Set Table ACL - No\n\nEntity Operations\n\n- Query Entities - Flavor A (QueryEntity by PartKey/RowKey) - Yes\n- Query Entities - Flavor B (QueryEntities by OData $filter) - No\n- Insert Entity - Yes\n- Update Entity - Yes\n- Merge Entity - Yes\n- Delete Entity - Yes\n- Insert or Replace Entity - Yes\n- Insert or Merge Entity - Yes\n\nFile Service - 0/19 - N/A\n-----------------------------------------------------------\n\nAccount Operations\n\n- List Shares - No\n\nShare Operations\n\n- Create Share - No\n- Get Share Properties - No\n- Get Share Metadata - No\n- Set Share Metadata - No\n- Delete Share - No\n\nDirectory Operations\n\n- List Directories and Files - No\n- Create Directory - No\n- Get Directory Properties - No\n- Delete Directory - No\n\nFile Operations\n\n- Create File - No\n- Get File - No\n- Get File Properties - No\n- Set File Properties - No\n- Put Range - No\n- List Ranges - No\n- Get File Metadata - No\n- Set File Metadata - No\n- Delete File - No\n\n\nMotivation\n====================\n\n\u003cblockquote\u003eAsk me about the Azure Storage SDK...\u003c/blockquote\u003e\n\nAs I watched (and lived through, and tried to figure out) the evolution of the official\nAzure Storage SDK for .Net, I've continued to be annoyed by several things:\n\n1) Pseudo-State: The attempted object-ification of a REST service (adding state where there is none)\n\n2) Dueling Documentation: The mismatch in terminology and operations between the API and SDK\n\n3) Emulator Support: The inclusion of features that cannot be used for local development due to\n   lack of support in the emulator\n\n4) Information: The generic way exceptions are handled and thrown compared to the detail available from the API\n   _Note: There is extended info in the Storage SDK exception, you just have to dig to find it and it has moved\n          a couple times_\n\n5) Information: The lack of information about failed retries\n\n6) Readability.\n\n7) Testability: There is not a clear interface to mock or fake for tests.\n\n8) Testability: The internal tests for the SDK use logic to mangle HTTP requests, are long and rambling, and have\n   been removed from some versions (3.0).\n\nSome of these have improved with the later versions and with some of the tracing methods in later versions, but\nsome have remained underlying concepts (such as trying to use stateful objects to represent a service).\n\nI've written an implementation of the Azure API before, but I wanted to start writing one that:\n\n1) Matches the API. The documentation should exist only to describe the naming gap between the two, the few\n   places where the local library diverges from the API, how to setup things like the storage settings and error\n   handling, and how exceptions map to the API.\n\n2) Treats the service as a service. If you want to consume the service and add some pseudo form of state, do so.\n   The SDK should not force that on you.\n\n3) Provides every scrap of error information possible to help the developer or maintainer do their job.\n\n4) Provides underlying information about retry rates and retries that were later succesful\n\n5) Matches the API internally as well. Finding the implementation details of an API operation should not require\n   opening 15 files.\n\n6) Has a clear set of tests that serves almost as a bullet point list of how the API works for each operation\n\n7) Is easily mockable for unit testing code that interacts with the library.\n\n8) Does not force you to use asynchronous patterns throughout your codebase (except where the API requires them)\n   _Note: The standard Storage SDK doesn't force this, but many other libraries these days use only Async methods,\n          assuming that converting an entire existing application to async/await is an easy process_\n\n9) Requires no magic. There are no exposed properties that are only correct after another call is made, no\n   enumerated values that are not enums, no properties that actually call the API behind the scenes,  and when a\n   parameter has character or length restrictions, the library should be smart enough to tell you about them.\n\nCurrently retries are only communicated upward at the end of a failed operation rather than providing hooks to\ngather information on each individual failed retry. Restrictions on length and character usage have also not been\nadded yet.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftarwn%2FBasicAzureStorageSDK","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftarwn%2FBasicAzureStorageSDK","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftarwn%2FBasicAzureStorageSDK/lists"}