Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/devlooped/tablestorage
Repository pattern with POCO object support for storing to Azure / Cosmos DB Table Storage
https://github.com/devlooped/tablestorage
azure cosmosdb storage table
Last synced: 2 months ago
JSON representation
Repository pattern with POCO object support for storing to Azure / Cosmos DB Table Storage
- Host: GitHub
- URL: https://github.com/devlooped/tablestorage
- Owner: devlooped
- License: mit
- Created: 2021-05-04T17:45:47.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2024-10-23T16:56:05.000Z (3 months ago)
- Last Synced: 2024-10-23T23:54:46.278Z (3 months ago)
- Topics: azure, cosmosdb, storage, table
- Language: C#
- Homepage: https://clarius.org/TableStorage
- Size: 993 KB
- Stars: 26
- Watchers: 4
- Forks: 1
- Open Issues: 2
-
Metadata Files:
- Readme: readme.md
- Changelog: changelog.md
- Funding: .github/FUNDING.yml
- License: license.txt
Awesome Lists containing this project
README
 TableStorage
============
[](https://www.nuget.org/packages/Devlooped.TableStorage)
[](https://www.nuget.org/packages/Devlooped.TableStorage)
[](https://github.com/devlooped/TableStorage/blob/main/license.txt)
[](https://github.com/devlooped/TableStorage/actions)
*This project uses [SponsorLink](https://github.com/devlooped#sponsorlink)
and may issue IDE-only warnings if no active sponsorship is detected.*
Repository pattern with POCO object support for storing to Azure/CosmosDB Table Storage
> NOTE: This library is a thin wrapper around the latest Azure SDK v12+ for Table Storage,
> and uses [CloudStorageAccount](https://www.nuget.org/packages/Devlooped.CloudStorageAccount) which
> is a 100% backwards compatible implementation of the Azure SDK v11 `CloudStorageAccount` class.
## Usage
Given an entity like:
```csharp
public record Product(string Category, string Id)
{
public required string? Title { get; init; }
public double Price { get; init; }
public DateOnly CreatedAt { get; init; }
}
```
> NOTE: entity can have custom constructor, key properties can be read-only
> (Category and Id in this case for example), and it doesn't need to inherit
> from anything, implement any interfaces or use
> any custom attributes (unless you want to). As shown above, it can even be
> a simple record type.
The entity can be stored and retrieved with:
```csharp
var storageAccount = CloudStorageAccount.DevelopmentStorageAccount; // or production one
// We lay out the parameter names for clarity only.
var repo = TableRepository.Create(storageAccount,
tableName: "Products",
partitionKey: p => p.Category,
rowKey: p => p.Id);
var product = new Product("book", "1234")
{
Title = "Table Storage is Cool",
Price = 25.5,
};
// Insert or Update behavior (aka "upsert")
await repo.PutAsync(product);
// Enumerate all products in category "book"
await foreach (var p in repo.EnumerateAsync("book"))
Console.WriteLine(p.Price);
// Query books priced in the 20-50 range,
// project just title + price
await foreach (var info in from prod in repo.CreateQuery()
where prod.Price >= 20 and prod.Price <= 50
select new { prod.Title, prod.Price })
Console.WriteLine($"{info.Title}: {info.Price}");
// Get previously saved product.
Product saved = await repo.GetAsync("book", "1234");
// Delete product
await repo.DeleteAsync("book", "1234");
// Can also delete passing entity
await repo.DeleteAsync(saved);
```
Attributes can also be used to eliminate the need for lambdas altogether when
the entity storage layout is known at compile time:
```csharp
[Table("Products")]
public record Product([PartitionKey] string Category, [RowKey] string Id) ...
var storageAccount = CloudStorageAccount.DevelopmentStorageAccount;
// Everything discovered from attributes.
var repo = TableRepository.Create(storageAccount);
```
See the [Attributes](#attributes) section below for more details on how to use them.
If the product were books for example, it might make sense to partition by author.
In that case, you could use a `TableRepository` when saving:
```csharp
public record Book([RowKey] string ISBN, string Title, string Author, BookFormat Format, int Pages);
var repo = TableRepository.Create(storageAccount, "Books",
partitionKey: x => x.Author);
await repo.PutAsync(book);
```
> Note how you can mix and match attributes and explicit lambdas as needed.
> The latter takes precedence over the former.
And later on when listing/filtering books by a particular author, you can use
a `TablePartition` so all querying is automatically scoped to that author:
```csharp
var partition = TablePartition.Create(storageAccount, "Books",
partitionKey: "Rick Riordan");
// Get Rick Riordan books, only from Disney/Hyperion, with over 1000 pages
var query = from book in repo.CreateQuery()
where
book.ISBN.CompareTo("97814231") >= 0 &&
book.ISBN.CompareTo("97814232") < 0 &&
book.Pages >= 1000
select new { book.ISBN, book.Title };
```
Using table partitions is quite convenient for handling reference data too, for example.
Enumerating all entries in the partition wouldn't be something you'd typically do for
your "real" data, but for reference data, it could come in handy.
> NOTE: if access to the `Timestamp` managed by Azure Table Storage for the entity is needed,
> just declare a property with that name with either `DateTimeOffset`, `DateTime` or `string` type
> to read it.
Stored entities using `TableRepository` and `TablePartition` use individual columns for
properties, which makes it easy to browse the data (and query, as shown above!).
> NOTE: partition and row keys can also be typed as `Guid`
Document-based storage is also available via `DocumentRepository` and `DocumentPartition` if
you don't need the individual columns.
## Document Storage
The `DocumentRepository.Create` and `DocumentPartition.Create` factory methods provide access
to document-based storage, exposing the a similar API as column-based storage.
Document repositories cause entities to be persisted as a single document column, alongside type and version
information to handle versioning a the app level as needed.
The API is mostly the same as for column-based repositories (document repositories implement
the same underlying `ITableStorage` interface):
```csharp
public record Product(string Category, string Id)
{
public string? Title { get; init; }
public double Price { get; init; }
public DateOnly CreatedAt { get; init; }
}
var book = new Product("book", "9781473217386")
{
Title = "Neuromancer",
Price = 7.32
};
// Column-based storage
var repo = TableRepository.Create(
CloudStorageAccount.DevelopmentStorageAccount,
tableName: "Products",
partitionKey: p => p.Category,
rowKey: p => p.Id);
await repo.PutAsync(book);
// Document-based storage
var docs = DocumentRepository.Create(
CloudStorageAccount.DevelopmentStorageAccount,
tableName: "Documents",
partitionKey: p => p.Category,
rowKey: p => p.Id
serializer: [SERIALIZER]);
await docs.PutAsync(book);
```
> If not provided, the serializer defaults to the `System.Text.Json`-based `DocumentSerializer.Default`.
The resulting differences in storage can be seen in the following screenshots of the
[Azure Storage Explorer](https://azure.microsoft.com/en-us/features/storage-explorer/):
The `Type` column persisted in the documents table is the `Type.FullName` of the persisted entity, and the
`Version` is the `[Major].[Minor]` of its assembly, which could be used for advanced data migration scenarios.
The major and minor version components are also provided as individual columns for easier querying
by various version ranges, using `IDocumentRepository.EnumerateAsync(predicate)`.
In addition to the default built-in JSON plain-text based serializer (which uses the
[System.Text.Json](https://www.nuget.org/packages/system.text.json) package), there are other
alternative serializers for the document-based repository, including various binary serializers
which will instead persist the document as a byte array:
[](https://www.nuget.org/packages/Devlooped.TableStorage.Newtonsoft) [](https://www.nuget.org/packages/Devlooped.TableStorage.Bson) [](https://www.nuget.org/packages/Devlooped.TableStorage.MessagePack) [](https://www.nuget.org/packages/Devlooped.TableStorage.Protobuf)
You can pass the serializer to use to the factory method as follows:
```csharp
var repo = TableRepository.Create(...,
serializer: [JsonDocumentSerializer|BsonDocumentSerializer|MessagePackDocumentSerializer|ProtobufDocumentSerializer].Default);
```
> NOTE: when using alternative serializers, entities might need to be annotated with whatever
> attributes are required by the underlying libraries.
## Attributes
If you want to avoid using strings with the factory methods, you can also annotate the
entity type to modify the default values used:
* `[Table("tableName")]`: class-level attribute to change the default when no value is provided
* `[PartitionKey]`: annotates the property that should be used as the partition key
* `[RowKey]`: annotates the property that should be used as the row key.
Values passed to the factory methods override declarative attributes.
For the products example above, your record entity could be:
```csharp
[Table("Products")]
public record Product([PartitionKey] string Category, [RowKey] string Id)
{
public string? Title { get; init; }
public double Price { get; init; }
}
```
And creating the repository wouldn't require any arguments besides the storage account:
```csharp
var repo = TableRepository.Create(CloudStorageAccount.DevelopmentStorageAccount);
```
In addition, if you want to omit a particular property from persistence, you can annotate
it with `[Browsable(false)]` and it will be skipped when persisting and reading the entity.
## TableEntity Support
Since these repository APIs are quite a bit more intuitive than working directly against a
`TableClient`, you might want to retrieve/enumerate entities just by their built-in `TableEntity`
properties, like `PartitionKey`, `RowKey`, `Timestamp` and `ETag`. For this scenario, we
also support creating `ITableRepository` and `ITablePartition`
by using the factory methods `TableRepository.Create(...)` and `TablePartition.Create(...)`
without a (generic) entity type argument.
For example, given you know all `Region` entities saved in the example above, use the region `Code`
as the `RowKey`, you could simply enumerate all regions without using the `Region` type at all:
```csharp
var account = CloudStorageAccount.DevelopmentStorageAccount; // or production one
var repo = TablePartition.Create(storageAccount,
tableName: "Reference",
partitionKey: "Region");
// Enumerate all regions within the partition as plain TableEntities
await foreach (TableEntity region in repo.EnumerateAsync())
Console.WriteLine(region.RowKey);
```
You can access and add additional properties by just using the entity indexer, which you can
later persist by calling `PutAsync`:
```csharp
await repo.PutAsync(
new TableEntity("Book", "9781473217386")
{
["Title"] = "Neuromancer",
["Price"] = 7.32
});
var entity = await repo.GetAsync("Book", "9781473217386");
Assert.Equal("Neuromancer", entity["Title"]);
Assert.Equal(7.32, (double)entity["Price"]);
```
## Installation
```
> Install-Package Devlooped.TableStorage
```
All packages also come in source-only versions, if you want to avoid an additional assembly dependency:
```
> Install-Package Devlooped.TableStorage.Source
```
The source-only packages includes all types with the default visibility (internal), so you can decide
what types to make public by declaring a partial class with the desired visibility. To make them all
public, for example, you can include the same [Visibility.cs](https://github.com/devlooped/TableStorage/blob/main/src/TableStorage/Visibility.cs)
that the compiled version uses.
## Dogfooding
[](https://pkg.kzu.io/index.json)
[](https://github.com/devlooped/TableStorage/actions)
We also produce CI packages from branches and pull requests so you can dogfood builds as quickly as they are produced.
The CI feed is `https://pkg.kzu.io/index.json`.
The versioning scheme for packages is:
- PR builds: *42.42.42-pr*`[NUMBER]`
- Branch builds: *42.42.42-*`[BRANCH]`.`[COMMITS]`
# Sponsors
[](https://github.com/clarius)
[](https://github.com/KirillOsenkov)
[](https://github.com/MFB-Technologies-Inc)
[](https://github.com/torutek-gh)
[](https://github.com/drivenet)
[](https://github.com/Keflon)
[](https://github.com/tbolon)
[](https://github.com/kfrancis)
[](https://github.com/twenzel)
[](https://github.com/unoplatform)
[](https://github.com/dansiegel)
[](https://github.com/rbnswartz)
[](https://github.com/jfoshee)
[](https://github.com/Mrxx99)
[](https://github.com/eajhnsn1)
[](https://github.com/IxTechnologies)
[](https://github.com/davidjenni)
[](https://github.com/Jonathan-Hickey)
[](https://github.com/akunzai)
[](https://github.com/jakobt)
[](https://github.com/seanalexander)
[](https://github.com/tinohager)
[](https://github.com/ploeh)
[](https://github.com/KenBonny)
[](https://github.com/SimonCropp)
[](https://github.com/agileworks-eu)
[](https://github.com/sorahex)
[](https://github.com/arsdragonfly)
[](https://github.com/vezel-dev)
[](https://github.com/ChilliCream)
[](https://github.com/4OTC)
[](https://github.com/v-limo)
[](https://github.com/sponsors/devlooped)
[Learn more about GitHub Sponsors](https://github.com/sponsors)