{"id":20812542,"url":"https://github.com/superstreamlabs/memphis.net","last_synced_at":"2025-05-07T10:45:17.405Z","repository":{"id":103056400,"uuid":"574486015","full_name":"superstreamlabs/memphis.net","owner":"superstreamlabs","description":".NET client for Memphis. Memphis is an event processing platform","archived":false,"fork":false,"pushed_at":"2024-07-08T15:33:14.000Z","size":784,"stargazers_count":18,"open_issues_count":0,"forks_count":6,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-31T09:11:28.717Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://memphis.dev","language":"C#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/superstreamlabs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-12-05T12:23:05.000Z","updated_at":"2024-07-08T15:32:46.000Z","dependencies_parsed_at":"2023-11-26T10:23:28.646Z","dependency_job_id":"7e38be21-fd55-48a6-bde5-1f79336494c7","html_url":"https://github.com/superstreamlabs/memphis.net","commit_stats":null,"previous_names":["superstreamlabs/memphis.net"],"tags_count":22,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/superstreamlabs%2Fmemphis.net","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/superstreamlabs%2Fmemphis.net/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/superstreamlabs%2Fmemphis.net/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/superstreamlabs%2Fmemphis.net/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/superstreamlabs","download_url":"https://codeload.github.com/superstreamlabs/memphis.net/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252862832,"owners_count":21815925,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-17T20:55:22.898Z","updated_at":"2025-05-07T10:45:17.379Z","avatar_url":"https://github.com/superstreamlabs.png","language":"C#","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ca href=\"![Github (4)](https://github.com/memphisdev/memphis-terraform/assets/107035359/a5fe5d0f-22e1-4445-957d-5ce4464e61b1)\"\u003e[![Github (4)](https://github.com/memphisdev/memphis-terraform/assets/107035359/a5fe5d0f-22e1-4445-957d-5ce4464e61b1)](https://memphis.dev)\u003c/a\u003e\n\u003cp align=\"center\"\u003e\n  Please pay attention that Memphis.dev is no longer supported officially by the Superstream team (formerly Memphis.dev ) and was released to the public.\u003cbr/\u003e\n  \u003cp align=\"center\"\u003e\n\u003ca href=\"https://memphis.dev/discord\"\u003e\u003cimg src=\"https://img.shields.io/discord/963333392844328961?color=6557ff\u0026label=discord\" alt=\"Discord\"\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/memphisdev/memphis/issues?q=is%3Aissue+is%3Aclosed\"\u003e\u003cimg src=\"https://img.shields.io/github/issues-closed/memphisdev/memphis?color=6557ff\"\u003e\u003c/a\u003e \n  \u003cimg src=\"https://img.shields.io/npm/dw/memphis-dev?color=ffc633\u0026label=installations\"\u003e\n\u003ca href=\"https://github.com/memphisdev/memphis/blob/master/CODE_OF_CONDUCT.md\"\u003e\u003cimg src=\"https://img.shields.io/badge/Code%20of%20Conduct-v1.0-ff69b4.svg?color=ffc633\" alt=\"Code Of Conduct\"\u003e\u003c/a\u003e \n\u003cimg alt=\"GitHub release (latest by date)\" src=\"https://img.shields.io/github/v/release/memphisdev/memphis?color=61dfc6\"\u003e\n\u003cimg src=\"https://img.shields.io/github/last-commit/memphisdev/memphis?color=61dfc6\u0026label=last%20commit\"\u003e\n\u003c/p\u003e\n\n\u003cdiv align=\"center\"\u003e\n  \n  \u003ch4\u003e\n    \n**[Memphis.dev](https://memphis.dev)** is a highly scalable, painless, and effortless data streaming platform.\u003cbr\u003e\nMade to enable developers and data teams to collaborate and build\u003cbr\u003e\nreal-time and streaming apps fast.\n\n  \u003c/h4\u003e\n  \n\u003c/div\u003e\n\n## Installation\n\n```sh\n dotnet add package Memphis.Client -v ${MEMPHIS_CLIENT_VERSION}\n```\n\n## Update\n\n```sh\nUpdate-Package Memphis.Client\n```\n\n## Importing\n\n```c#\nusing Memphis.Client;\n```\nFirst, a connection to Memphis must be made:\n\n```c#\nusing Memphis.Client;\n\n// Connecting to the broker\nvar options = MemphisClientFactory.GetDefaultOptions();\noptions.Host = \"aws-us-east-1.cloud.memphis.dev\";\noptions.AccountId = int.Parse(Environment.GetEnvironmentVariable(\"memphis_account_id\"));\noptions.Username = \"test_user\";\noptions.Password = Environment.GetEnvironmentVariable(\"memphis_pass\");\n\nvar memphisClient = await MemphisClientFactory.CreateClient(options);\n```\n\nThen, to produce a message, call the `memphisClient.ProduceAsync` function or create a producer and call its `producer.ProduceAsync` function:\n\n```C#\nMessage message = new()\n{\n    Hello = \"World!\"\n};\n\nvar headers = new NameValueCollection();\n\nvar msgBytes = Encoding.UTF8.GetBytes(JsonSerializer.Serialize(message));\nawait memphisClient.ProduceAsync(new Memphis.Client.Producer.MemphisProducerOptions\n    {\n        StationName = \"test_station\",\n        ProducerName = \"producer\"\n    },\n    msgBytes,\n    headers);\n\npublic class Message\n{\n    public string Hello { get; set; }\n}\n```\n\nLastly, to consume this message, call the `memphisClient.FetchMessages` function or create a consumer and call its `consumer.Fetch` function:\n\n```C#\nvar messages = await memphisClient.FetchMessages(new Memphis.Client.Consumer.FetchMessageOptions\n    {\n        StationName = \"test_station\",\n        ConsumerName = \"consumer\",\n        Prefetch = false\n    });\n\nforeach (MemphisMessage message in messages)\n{\n    var messageData = message.GetData();\n    var messageOBJ = JsonSerializer.Deserialize\u003cMessage\u003e(messageData);\n\n    // Do something with the message object here\n\n    message.Ack();\n}\n```\n\n\u003e Remember to call `memphisClient.Dispose()` to close the connection!\n\n\n### Connecting to Memphis\n\nThe createClient method in the Memphis class allows for the connection to Memphis. Connecting to Memphis (cloud or open-source) will be needed in order to use any of the other functionality of the Memphis class. Upon connection, all of Memphis' features are available.\n\nFirst, we need to create or use default `ClientOptions` and then connect to Memphis by using `MemphisClientFactory.CreateClient(ClientOptions opts)`.\n\n```c#\ntry\n{\n    var options = MemphisClientFactory.GetDefaultOptions();\n    options.Host = \"\u003cbroker-address\u003e\";\n    options.Username = \"\u003cusername\u003e\";\n    options.ConnectionToken = \"\u003cbroker-token\u003e\"; // you will get it on broker creation\n    options.AccountId = \u003caccount-id\u003e; // You can find it on the profile page in the Memphis UI. This field should be set only on the cloud version of Memphis, otherwise it will be ignored\n    options.MaxReconnect = \u003cmax-reconnect\u003e; // Gets or sets the maximum number of times connection will attempt to reconnect. To reconnect indefinitely set this value to -1. The default value is -1.\n    var memphisClient = await MemphisClientFactory.CreateClient(options);\n    ...\n}\ncatch (Exception ex)\n{\n    Console.Error.WriteLine(\"Exception: \" + ex.Message);\n    Console.Error.WriteLine(ex);\n}\n```\n\nA password-based connection would look like this (using the defualt root memphis login with Memphis open-source):\n\n```csharp\ntry\n{\n    var options = MemphisClientFactory.GetDefaultOptions();\n    options.Host = \"localhost\";\n    options.Username = \"root\";\n    options.Password = \"memphis\";  \n    var memphisClient = await MemphisClientFactory.CreateClient(options);\n}\ncatch (Exception ex) {\n    // handle exception\n}\n```\n\nIf you wanted to connect to Memphis cloud instead, simply add your account ID and change the host. The host and account_id can be found on the Overview page in the Memphis cloud UI under your name at the top. Here is an example to connecting to a cloud broker that is located in US East:  \n\n```csharp\ntry\n{\n    var options = MemphisClientFactory.GetDefaultOptions();\n    options.Host = \"aws-us-east-1.cloud.memphis.dev\";\n    options.Username = \"my_client_username\";\n    options.Password = \"my_client_password\";  \n    options.AccountId = 123456789;\n    var memphisClient = await MemphisClientFactory.CreateClient(options);\n}\ncatch (Exception ex) {\n    // handle exception\n}\n```\n\nIt is possible to use a token-based connection to memphis as well, where multiple users can share the same token to connect to memphis. Here is an example of using memphis.connect with a token:\n\n```csharp\ntry\n{\n    var options = MemphisClientFactory.GetDefaultOptions();\n    options.Host = \"localhost\";\n    options.Username = \"root\";\n    options.ConnectionToken = \"Token\";  \n    var memphisClient = await MemphisClientFactory.CreateClient(options);\n}\ncatch (Exception ex) {\n    // handle exception\n}\n```\n\nThe token will be presented when creating new users. \n\nMemphis needs to be configured to use token based connection. See the [docs](https://docs.memphis.dev/memphis/memphis-broker/concepts/security) for help doing this.\n\nA TLS based connection would look like this:\n\n```csharp\ntry\n{\n    var options = MemphisClientFactory.GetDefaultOptions();\n    options.Host = \"localhost\";\n    options.Username = \"root\";\n    options.Tls = new TlsOptions(\"tlsFileName\");\n    var memphisClient = await MemphisClientFactory.CreateClient(options);\n}\ncatch (Exception ex)\n{\n    // handle exception\n}\n```\n\nMemphis needs to configured for these use cases. To configure memphis to use TLS see the [docs](https://docs.memphis.dev/memphis/open-source-installation/kubernetes/production-best-practices#memphis-metadata-tls-connection-configuration). \n\n### Disconnecting from Memphis\n\nTo disconnect from Memphis, call `Dispose()` on the `MemphisClient`.\n\n```c#\nmemphisClient.Dispose()\n```\n### Creating a Station\n\nStations are distributed units that store messages. Producers add messages to stations and Consumers take messages from them. Each station stores messages until their retention policy causes them to either delete the messages or move them to [remote storage](https://docs.memphis.dev/memphis/integrations-center/storage/s3-compatible). \n\n**A station will be automatically created for the user when a consumer or producer is used if no stations with the given station name exist.**\u003cbr\u003e\u003cbr\u003e\n_If the station trying to be created exists when this function is called, nothing will change with the exisitng station_\n\n```c#\ntry\n{\n    // First: creating Memphis client\n    var options = MemphisClientFactory.GetDefaultOptions();\n    options.Host = \"\u003cmemphis-host\u003e\";\n    options.Username = \"\u003cusername\u003e\";\n    options.Password = \"\u003cpassword\u003e\";\n    options.AccountId = \u003caccount-id\u003e; // You can find it on the profile page in the Memphis UI. This field should be set only on the cloud version of Memphis, otherwise it will be ignored\n    var client = await MemphisClientFactory.CreateClient(options);\n    \n    // Second: creaing Memphis station\n    var station = await client.CreateStation(\n        stationOptions: new StationOptions()\n        {\n            Name = \"\u003cstation-name\u003e\",\n            RetentionType = RetentionTypes.MAX_MESSAGE_AGE_SECONDS,\n            RetentionValue = 3600,\n            StorageType = StorageTypes.DISK,\n            Replicas = 1,\n            IdempotencyWindowMs = 0,\n            SendPoisonMessageToDls = true,\n            SendSchemaFailedMessageToDls = true,\n            PartitionsNumber = 3, // defaults to 1\n            DlsStation = \"\u003cdls-station\u003e\" // If DlsStation is set, then DLS events will be sent to selected station as well. The default value is \"\" (no DLS station).\n        });\n}\ncatch (Exception ex)\n{\n    Console.Error.WriteLine(\"Exception: \" + ex.Message);\n    Console.Error.WriteLine(ex);\n}\n```\n\nThe CreateStation method is used to create a station. Using the different options available, one can programically create many different types of stations. The Memphis UI can also be used to create stations to the same effect. \n\nA minimal example, using all default values would simply create a station with the given name:\n\n```csharp\ntry\n{\n    var options = MemphisClientFactory.GetDefaultOptions();\n    options.Host = \"localhost\";\n    options.Username = \"root\";\n    options.Password = \"memphis\";\n    var memphisClient = await MemphisClientFactory.CreateClient(options);\n\n    var station = await memphisClient.CreateStation(\n        stationOptions: new StationOptions\n        {\n            Name = \"MyNewStation\"\n        }\n    );\n}\ncatch (Exception ex)\n{\n    // handle exception\n}\n```\n\nTo change what criteria the station uses to decide if a message should be retained in the station, change the retention type. The different types of retention are documented [here](https://github.com/memphisdev/memphis.net#retention-types) in the dotnet README. \n\nThe unit of the rentention value will vary depending on the RetentionTypes. The [previous link](https://github.com/memphisdev/memphis.net#retention-types) also describes what units will be used. \n\nHere is an example of a station which will only hold up to 10 messages:\n\n```csharp\ntry\n{\n    var options = MemphisClientFactory.GetDefaultOptions();\n    options.Host = \"localhost\";\n    options.Username = \"root\";\n    options.Password = \"memphis\";\n    var memphisClient = await MemphisClientFactory.CreateClient(options);\n\n    var station = await memphisClient.CreateStation(\n        stationOptions: new StationOptions\n        {\n            Name = \"MyNewStation\",\n            RetentionType = RetentionTypes.MESSAGES,\n            RetentionValue = 10\n        }\n    );  \n}\ncatch (Exception ex)\n{\n    // handle exception\n}\n```\n\nMemphis stations can either store Messages on disk or in memory. A comparison of those types of storage can be found [here](https://docs.memphis.dev/memphis/memphis-broker/concepts/storage-and-redundancy#tier-1-local-storage).\n\nHere is an example of how to create a station that uses Memory as its storage type:\n\n```csharp\ntry\n{\n    var options = MemphisClientFactory.GetDefaultOptions();\n    options.Host = \"localhost\";\n    options.Username = \"root\";\n    options.Password = \"memphis\";\n    var memphisClient = await MemphisClientFactory.CreateClient(options);\n\n    var station = await memphisClient.CreateStation(\n        stationOptions: new StationOptions\n        {\n            Name = \"MyNewStation\",\n            StorageType = StorageTypes.MEMORY\n        }\n    );\n}\ncatch (Exception ex) {\n    // handle exception\n}\n```\n\nIn order to make a station more redundant, replicas can be used. Read more about replicas [here](https://docs.memphis.dev/memphis/memphis-broker/concepts/storage-and-redundancy#replicas-mirroring). Note that replicas are only available in cluster mode. Cluster mode can be enabled in the [Helm settings](https://docs.memphis.dev/memphis/open-source-installation/kubernetes/1-installation#appendix-b-helm-deployment-options) when deploying Memphis with Kubernetes.\n\nHere is an example of creating a station with 3 replicas:\n\n```csharp\ntry\n{\n    var options = MemphisClientFactory.GetDefaultOptions();\n    options.Host = \"localhost\";\n    options.Username = \"root\";\n    options.Password = \"memphis\";\n    var memphisClient = await MemphisClientFactory.CreateClient(options);\n\n    var station = await memphisClient.CreateStation(\n        stationOptions: new StationOptions\n        {\n            Name = \"MyNewStation\",\n            Replicas = 3\n        }\n    );\n}\ncatch (Exception ex)\n{\n    // handle exception\n}\n```\n\nIdempotency defines how Memphis will prevent duplicate messages from being stored or consumed. The duration of time the message ID's will be stored in the station can be set with IdempotencyWindowMs. If the environment Memphis is deployed in has unreliably connection and/or a lot of latency, increasing this value might be desiriable. The default duration of time is set to two minutes. Read more about idempotency [here](https://docs.memphis.dev/memphis/memphis-broker/concepts/idempotency).\n\nHere is an example of changing the idempotency window to 3 seconds:\n\n```csharp\ntry\n{\n    var options = MemphisClientFactory.GetDefaultOptions();\n    options.Host = \"localhost\";\n    options.Username = \"root\";\n    options.Password = \"memphis\";\n    var memphisClient = await MemphisClientFactory.CreateClient(options);\n\n    var station = await memphisClient.CreateStation(\n        stationOptions: new StationOptions\n        {\n            Name = \"MyNewStation\",\n            IdempotenceWindowMs = 180_000\n        }\n    );\n}\ncatch (Exception ex)\n{\n    // handle exception\n}\n```\n\nThe schema name is used to set a schema to be enforced by the station. The default value of \"\" ensures that no schema is enforced. Here is an example of changing the schema to a defined schema in schemaverse called \"SensorLogs\":\n\n```csharp\ntry\n{\n    var options = MemphisClientFactory.GetDefaultOptions();\n    options.Host = \"localhost\";\n    options.Username = \"root\";\n    options.Password = \"memphis\";\n    var memphisClient = await MemphisClientFactory.CreateClient(options);\n\n    var station = await memphisClient.CreateStation(\n        stationOptions: new StationOptions\n        {\n            Name = \"MyNewStation\",\n            SchemaName = \"SensorLogs\"\n        }\n    );\n}\ncatch (Exception ex)\n{\n    // handle exception\n}\n```\n\nThere are two options for sending messages to the [dead-letter station(DLS)](https://docs.memphis.dev/memphis/memphis-broker/concepts/dead-letter#terminology). These are SendPoisonMessageToDls and SendSchemaFailedMessageToDls. \n\nHere is an example of sending poison messages to the DLS but not messages which fail to conform to the given schema.\n\n```csharp\n    try\n    {\n        var options = MemphisClientFactory.GetDefaultOptions();\n        options.Host = \"localhost\";\n        options.Username = \"root\";\n        options.Password = \"memphis\";\n        var memphisClient = await MemphisClientFactory.CreateClient(options);\n\n        var station = await memphisClient.CreateStation(\n            stationOptions: new StationOptions\n            {\n                Name = \"MyNewStation\",\n                SendPoisonMessageToDls = true,\n                SendSchemaFailedMessageToDls = false\n            }\n        );\n    }\n    catch (Exception ex)\n    {\n        // handle exception\n    }\n```\n\nWhen either of the DLS flags are set to True, a station can also be set to handle these events. To set a station as the station to where schema failed or poison messages will be set to, use the DlsStation StationOptions:\n\n```csharp\ntry\n{\n    var options = MemphisClientFactory.GetDefaultOptions();\n    options.Host = \"localhost\";\n    options.Username = \"root\";\n    options.Password = \"memphis\";\n    var memphisClient = await MemphisClientFactory.CreateClient(options);\n\n    var station = await memphisClient.CreateStation(\n        stationOptions: new StationOptions\n        {\n            Name = \"MyNewStation\",\n            SendPoisonMessageToDls = true,\n            SendSchemaFailedMessageToDls = false,\n            // DlsStation = \"DeadLetterMessageStation\" // Coming soon\n        }\n    );\n}\ncatch (Exception ex)\n{\n    // handle exception\n}\n```\n\nWhen the retention value is met, Mempihs by default will delete old messages. If tiered storage is setup, Memphis can instead move messages to tier 2 storage. Read more about tiered storage [here](https://docs.memphis.dev/memphis/memphis-broker/concepts/storage-and-redundancy#storage-tiering). Enable this setting with the option provided:\n\n```csharp\ntry\n{\n    var options = MemphisClientFactory.GetDefaultOptions();\n    options.Host = \"localhost\";\n    options.Username = \"root\";\n    options.Password = \"memphis\";\n    var memphisClient = await MemphisClientFactory.CreateClient(options);\n\n    var station = await memphisClient.CreateStation(\n        stationOptions: new StationOptions\n        {\n            Name = \"MyNewStation\",\n            TieredStorageEnabled = true\n        }\n    );\n}\ncatch (Exception ex)\n{\n    // handle exception\n}\n```\n\n[Partitioning](https://docs.memphis.dev/memphis/memphis-broker/concepts/station#partitions) might be useful for a station. To have a station partitioned, simply change the partitions number:\n\n```csharp\ntry\n{\n    var options = MemphisClientFactory.GetDefaultOptions();\n    options.Host = \"localhost\";\n    options.Username = \"root\";\n    options.Password = \"memphis\";\n    var memphisClient = await MemphisClientFactory.CreateClient(options);\n\n    var station = await memphisClient.CreateStation(\n        stationOptions: new StationOptions\n        {\n            Name = \"MyNewStation\",\n            PartitionsNumber = 3\n        }\n    );\n}\ncatch (Exception ex)\n{\n    // handle exception\n}\n```\n\n\n### Retention types\n\nRetention types define the methodology behind how a station behaves with its messages. Memphis currently supports the following retention types:\n\n```c#\nRetentionTypes.MAX_MESSAGE_AGE_SECONDS\n```\nWhen the retention type is set to MAX_MESSAGE_AGE_SECONDS, messages will persist in the station for the number of seconds specified in the retention_value. \n\n```c#\nRetentionTypes.MESSAGES\n```\nWhen the retention type is set to MESSAGES, the station will only hold up to retention_value messages. The station will delete the oldest messsages to maintain a retention_value number of messages.\n\n```c#\nRetentionTypes.BYTES\n```\nWhen the retention type is set to BYTES, the station will only hold up to retention_value BYTES. The oldest messages will be deleted in order to maintain at maximum retention_vlaue BYTES in the station.\n\n```c#\nRetentionTypes.ACK_BASED\n```\nWhen the retention type is set to ACK_BASED, messages in the station will be deleted after they are acked by all subscribed consumer groups.\n\n### Retention Values\n\nThe unit of the `retention_value` changes depending on the `retention_type` specified. \n\nAll retention values are of type `int`. The following units are used based on the respective retention type:\n\n`RetentionTypes.MAX_MESSAGE_AGE_SECONDS` is represented **in seconds**,\u003cbr\u003e\n`RetentionTypes.MESSAGES` is a **number of messages**,\u003cbr\u003e\n`RetentionTypes.BYTES` is a **number of bytes**,\u003cbr\u003e\nWith `RetentionTypes.ACK_BASED` the `RetentionType` is ignored. \n\n### Storage Types\n\nMemphis currently supports the following types of messages storage:\n\n```c#\nStorageTypes.DISK\n```\nWhen storage is set to DISK, messages are stored on disk.\n\n```c#\nStorageTypes.MEMORY\n```\nWhen storage is set to MEMORY, messages are stored in the system memory.\n\n### Destroying a Station\n\nDestroying a station will remove all its resources (including producers and consumers).\n\n```c#\nstation.DestroyAsync()\n```\n\n### Creating a new Schema\nIn case schema is already exist a new version will be created\n\n```c#\nawait client.CreateSchema(\"\u003cschema-name\u003e\", \"\u003cschema-type\u003e\", \"\u003cschema-file-path\u003e\")\n```\n\n### Enforcing a Schema on an Existing Station\n\n```c#\nawait client.EnforceSchema(stationName: \"\u003cstation-name\u003e\", schemaName: \"\u003cschema-name\u003e\");\n```\n\n### Deprecated - Attaching Schema\n\nThe `AttachSchema` method is depricated, use `EnforceSchema` instead.\n```c#\nawait client.AttachSchema(stationName: \"\u003cstation-name\u003e\", schemaName: \"\u003cschema-name\u003e\");\n```\n\n### Detaching a Schema from Station\n```c#\nawait client.DetachSchema(stationName: station.Name);\n```\n\n\n### Produce and Consume messages\n\nThe most common client operations are using `produce` to send messages and `consume` to\nreceive messages.\n\nMessages are published to a station with a Producer and consumed from it by a Consumer. \n\nConsumers are poll based and consume all the messages in a station. Consumers can also be grouped into consumer groups. When consuming with a consumer group, all consumers in the group will receive each message.\n\nMemphis messages are payload agnostic. Payloads are always `byte[]`s.\n\nIn order to stop getting messages, you have to call `consumer.Dispose()`. Destroy will terminate the consumer even if messages are currently being sent to the consumer.\n\nIf a station is created with more than one partition, producing to and consuming from the station will happen in a round robin fashion. \n\n### Creating a Producer\n\n```c#\ntry\n{\n   // First: creating Memphis client\n    var options = MemphisClientFactory.GetDefaultOptions();\n    options.Host = \"\u003cmemphis-host\u003e\";\n    options.Username = \"\u003cusername\u003e\";\n    options.Password = \"\u003cpassword\u003e\";\n    var client = await MemphisClientFactory.CreateClient(options);\n\n    // Second: creating the Memphis producer \n    var producer = await client.CreateProducer(new MemphisProducerOptions\n    {\n        StationName = \"\u003cmemphis-station-name\u003e\",\n        ProducerName = \"\u003cmemphis-producer-name\u003e\"\n    });\n}\ncatch (Exception ex)\n{\n    Console.Error.WriteLine(\"Exception: \" + ex.Message);\n    Console.Error.WriteLine(ex);\n}\n```\n\n### Producing a message\n\n```c#\nvar commonHeaders = new NameValueCollection();\ncommonHeaders.Add(\"key-1\", \"value-1\");\n\nawait producer.ProduceAsync(\n    message: Encoding.UTF8.GetBytes(text),\n    headers:commonHeaders\n);\n```\nNote:\nWhen producing to a station with more than one partition, the producer will produce messages in a Round Robin fashion between the different partitions.\n\nThe ProduceAsync method allows for the user to produce a message without discretely creating a producer. Because this creates a producer for every message, it is better to create a producer if many message need to be produced. \n\nFor message data formats see [here](https://docs.memphis.dev/memphis/memphis-schemaverse/formats/produce-consume). \n\nMessages produced by ProduceAsync run asyncronously by default. By using the AsyncProduce Option this can be set to produce messages syncronously, waiting for an ack after each message is produced. By default, messages are sent while still waiting for the ack of previously sent messages. This reduces preceived network latency and will allow for producers to produce more messages however may incur a loss in reliability. \n\nHere is an example of a ProduceAsync method call that waits up to one minute for an acknowledgement from memphis and produces messages syncronously:\n\n```csharp\ntry\n{\n    var options = MemphisClientFactory.GetDefaultOptions();\n    options.Host = \"localhost\";\n    options.Username = \"root\";\n    options.Password = \"memphis\";\n    var memphisClient = await MemphisClientFactory.CreateClient(options);\n\n    await memphisClient.ProduceAsync(\n        options: new MemphisProducerOptions\n        {\n            MaxAckTimeMs = 60_000,\n            StationName = \"MyStation\",\n            ProducerName = \"MyProducer\"\n        },\n        message: Encoding.UTF8.GetBytes(\"MyMessage\"),\n        asyncProduceAck: false\n    );\n}\ncatch (Exception ex)\n{\n    // handle exception\n}\n```\n\nAs discussed before in the station section, idempotency is an important feature of memphis. To achieve idempotency, an id must be assigned to messages that are being produced. Use the messageId parameter for this purpose.\n\n```csharp\ntry\n{\n    var options = MemphisClientFactory.GetDefaultOptions();\n    options.Host = \"localhost\";\n    options.Username = \"root\";\n    options.Password = \"memphis\";\n    var memphisClient = await MemphisClientFactory.CreateClient(options);\n\n    await memphisClient.ProduceAsync(\n        options: new MemphisProducerOptions\n        {\n            MaxAckTimeMs = 60_000,\n            StationName = \"MyStation\",\n            ProducerName = \"MyProducer\"\n        },    \n        message: Encoding.UTF8.GetBytes(\"MyMessage\"),\n        messageId: \"UniqueMessageID\"\n    );\n}\ncatch (Exception ex)\n{\n    // handle exception\n}\n```\n\nTo add message headers to the message, use the headers parameter. Headers can help with observability when using certain 3rd party to help monitor the behavior of memphis. See [here](https://docs.memphis.dev/memphis/memphis-broker/comparisons/aws-sqs-vs-memphis#observability) for more details.\n\n```csharp\ntry\n{\n    var options = MemphisClientFactory.GetDefaultOptions();\n    options.Host = \"localhost\";\n    options.Username = \"root\";\n    options.Password = \"memphis\";\n    var memphisClient = await MemphisClientFactory.CreateClient(options);\n\n    var headers = new NameValueCollection\n    {\n        { \"trace_header\", \"track_me_123\" }\n    };\n\n    await memphisClient.ProduceAsync(\n    options: new MemphisProducerOptions\n    {\n        MaxAckTimeMs = 60_000,\n        StationName = \"MyStation\",\n        ProducerName = \"MyProducer\"\n    },\n    message: Encoding.UTF8.GetBytes(\"MyMessage\"),\n    headers: headers\n    );\n}\ncatch (Exception ex)\n{\n    // handle exception\n}\n```\n\nMemphis can produce to a specific partition in a station. To do so, use the partitionKey parameter:\n\n```csharp\ntry\n{\n    var options = MemphisClientFactory.GetDefaultOptions();\n    options.Host = \"localhost\";\n    options.Username = \"root\";\n    options.Password = \"memphis\";\n    var memphisClient = await MemphisClientFactory.CreateClient(options);\n\n    await memphisClient.ProduceAsync(\n        options: new MemphisProducerOptions\n        {\n            MaxAckTimeMs = 60_000,\n            StationName = \"MyStation\",\n            ProducerName = \"MyProducer\"\n        },    \n        message: Encoding.UTF8.GetBytes(\"MyMessage\")\n        partitionKey: \"Partition3\"\n    );\n}\ncatch (Exception ex)\n{\n    // handle exception\n}\n```\n\n### Produce using partition number\nThe partition number will be used to produce messages to a spacific partition.\n\n```csharp\nawait producer.ProduceAsync(\n    message: Encoding.UTF8.GetBytes(text),\n    headers:commonHeaders, \n    partitionNumber:\u003cint\u003e // default is -1\n);\n```\n\n### Produce to multiple stations\n\nTo produce a message to multiple stations, use the `StationNames` property of the `MemphisProducerOptions`:\n\n```csharp\ntry\n{\n    // First: creating Memphis client\n    var options = MemphisClientFactory.GetDefaultOptions();\n    options.Host = \"\u003cmemphis-host\u003e\";\n    options.Username = \"\u003cusername\u003e\";\n    options.Password = \"\u003cpassword\u003e\";\n    var client = await MemphisClientFactory.CreateClient(options);\n\n    // Second: creating the Memphis producer \n    var producer = await client.CreateProducer(new MemphisProducerOptions\n    {\n        StationName = \"\u003cmemphis-station-name\u003e\",\n        ProducerName = \"\u003cmemphis-producer-name\u003e\",\n        StationNames = new string[] { \"\u003cmemphis-station-name-1\u003e\", \"\u003cmemphis-station-name-2\u003e\" }\n    });\n\n    // Third: sending the message\n    var headers = new NameValueCollection\n    {\n        { \"key-1\", \"value-1\" }\n    };\n    await producer.ProduceAsync(\n        message: Encoding.UTF8.GetBytes(\"Hello World!\"),\n        headers:headers\n    );\n}\ncatch (Exception ex)\n{\n    // handle exception\n}\n```\n\nIt is also possible to produce a message to multiple stations using the `ProduceAsync` method of the `MemphisClient`:\n\n```csharp\n// First: creating Memphis client\nvar options = MemphisClientFactory.GetDefaultOptions();\noptions.Host = \"\u003cmemphis-host\u003e\";\noptions.Username = \"\u003cusername\u003e\";\noptions.Password = \"\u003cpassword\u003e\";\n\nvar client = await MemphisClientFactory.CreateClient(options);\n\nvar headers = new NameValueCollection\n{\n    { \"trace_header\", \"track_me_123\" }\n};\n\n// Second: sending the message\nawait client.ProduceAsync(\n    options: new MemphisProducerOptions\n    {\n        MaxAckTimeMs = 60_000,\n        ProducerName = \"MyProducer\",\n        StationNames = new string[] { \"\u003cmemphis-station-name-1\u003e\", \"\u003cmemphis-station-name-2\u003e\" }\n    },\n    message: Encoding.UTF8.GetBytes(\"MyMessage\"),\n    headers: headers\n);\n```\n\n### Destroying a Producer\n\n```c#\nawait producer.DestroyAsync()\n```\n\n### Creating a Consumer\n\n```c#\ntry\n{\n    // First: creating Memphis client\n    var options = MemphisClientFactory.GetDefaultOptions();\n    options.Host = \"\u003cmemphis-host\u003e\";\n    options.Username = \"\u003cusername\u003e\";\n    options.Password = \"\u003cpassword\u003e\";\n    var client = await MemphisClientFactory.CreateClient(options);\n    \n    // Second: creaing Memphis consumer\n    var consumer = await client.CreateConsumer(new ConsumerOptions\n    {\n        StationName = \"\u003cstation-name\u003e\",\n        ConsumerName = \"\u003cconsumer-name\u003e\",\n        ConsumerGroup = \"\u003cconsumer-group-name\u003e\",\n    }); \n       \n}\ncatch (Exception ex)\n{\n    Console.Error.WriteLine(\"Exception: \" + ex.Message);\n    Console.Error.WriteLine(ex);\n}\n```\n\nNote:\nWhen consuming from a station with more than one partition, the consumer will consume messages in Round Robin fashion from the different partitions.\n\nUse the Memphis CreateConsumer method to create a Consumer. Consumers are used to pull messages from stations.\n\nHere is an example on how to create a consumer with all of the default options:\n\n```csharp\ntry\n{\n    var options = MemphisClientFactory.GetDefaultOptions();\n    options.Host = \"localhost\";\n    options.Username = \"root\";\n    options.Password = \"memphis\";\n    var memphisClient = await MemphisClientFactory.CreateClient(options);\n\n    var consumer = await memphisClient.CreateConsumer(new MemphisConsumerOptions\n    {\n        StationName = \"MyStation\",\n        ConsumerName = \"MyConsumer\"\n    });\n}\ncatch (Exception ex)\n{\n    // handle exception\n}handle exception\n}\n```\n\nTo create a consumer in a consumer group, add the ConsumerGroup MemphisConsumerOptions:\n\n```csharp\ntry\n{\n    var options = MemphisClientFactory.GetDefaultOptions();\n    options.Host = \"localhost\";\n    options.Username = \"root\";\n    options.Password = \"memphis\";\nvar memphisClient = await MemphisClientFactory.CreateClient(options);\n\n    var consumer = await memphisClient.CreateConsumer(new MemphisConsumerOptions\n    {\n        StationName = \"MyStation\",\n        ConsumerName = \"MyConsumer\",\n        ConsumerGroup = \"MyConsumerGroup1\"\n    });\n}\ncatch (Exception ex)\n{\n    // handle exception\n}\n```\n\nWhen using Consumer.consume, the consumer will continue to consume in an infinite loop. To change the rate at which the consumer polls, change the PullIntervalMs parameter:\n\n```csharp\ntry\n{\n    var options = MemphisClientFactory.GetDefaultOptions();\n    options.Host = \"localhost\";\n    options.Username = \"root\";\n    options.Password = \"memphis\";\n    var memphisClient = await MemphisClientFactory.CreateClient(options);\n\n    var consumer = await memphisClient.CreateConsumer(new MemphisConsumerOptions\n    {\n        StationName = \"MyStation\",\n        ConsumerName = \"MyConsumer\",\n        PullIntervalMs = 2_000\n    });\n}\ncatch (Exception ex)\n{\n    // handle exception\n}\n```\n\nEvery time the consumer polls, the consumer will try to take BatchSize number of elements from the station. However, sometimes there are not enough messages in the station for the consumer to consume a full batch. In this case, the consumer will continue to wait until either BatchSize messages are gathered or the time in milliseconds specified by BatchMaxTimeToWaitMs is reached. \n\nHere is an example of a consumer that will try to pull 100 messages from a station every 10 seconds while waiting up to 15 seconds for all messages to reach the consumer.\n\n```csharp\ntry\n{\n    var options = MemphisClientFactory.GetDefaultOptions();\n    options.Host = \"localhost\";\n    options.Username = \"root\";\n    options.Password = \"memphis\";\nvar memphisClient = await MemphisClientFactory.CreateClient(options);\n\n    var consumer = await memphisClient.CreateConsumer(new MemphisConsumerOptions\n    {\n        StationName = \"MyStation\",\n        ConsumerName = \"MyConsumer\",\n        PullIntervalMs = 10_000,\n        BatchSize = 100,\n        BatchMaxTimeToWaitMs = 1000\n    });\n}\ncatch (Exception ex)\n{\n    // handle exception\n}\n```\n\nThe MaxMsgDeliveries parameter allows the user how many messages the consumer is able to consume before consuming more.\n\n```csharp\ntry\n{\n    var options = MemphisClientFactory.GetDefaultOptions();\n    options.Host = \"localhost\";\n    options.Username = \"root\";\n    options.Password = \"memphis\";\n    var memphisClient = await MemphisClientFactory.CreateClient(options);\n\n    var consumer = await memphisClient.CreateConsumer(new MemphisConsumerOptions\n    {\n        StationName = \"MyStation\",\n        ConsumerName = \"MyConsumer\",\n        PullIntervalMs = 10_000,\n        BatchSize = 100,\n        BatchMaxTimeToWaitMs = 1000,\n        MaxMsgDeliveries = 2\n    });\n}\ncatch (Exception ex)\n{\n    // handle exception\n}\n```\n\n### Creating message handler for consuming a message\n\nTo configure message handler, use the `MessageReceived` event:\n\n```c#\nconsumer.MessageReceived += (sender, args) =\u003e\n{\n    if (args.Exception != null)\n    {\n        Console.Error.WriteLine(args.Exception);\n        return;\n    }\n\n    foreach (var msg in args.MessageList)\n    {\n        //print message itself\n        Console.WriteLine(\"Received data: \" + Encoding.UTF8.GetString(msg.GetData()));\n\n\n        // print message headers\n        foreach (var headerKey in msg.GetHeaders().Keys)\n        {\n            Console.WriteLine(\n                $\"Header Key: {headerKey}, value: {msg.GetHeaders()[headerKey.ToString()]}\");\n        }\n\n        Console.WriteLine(\"---------\");\n        msg.Ack();\n    }\n    Console.WriteLine(\"destroyed\");\n};\n```\n\n### Consuming a message\n\nThe consumer will try to fetch messages every _PullIntervalMs_ (that was given in Consumer's creation) and call the defined message handler.\n\n```c#\n await consumer.ConsumeAsync();\n```\n\n\n#### Consumer schema deserialization\nTo get messages deserialized, use `msg.GetDeserializedData()` or  `msg.GetDeserializedData\u003cT\u003e()`.  \n\n```csharp\nconsumer.MessageReceived += (sender, args) =\u003e\n{\n    if (args.Exception != null)\n    {\n        Console.Error.WriteLine(args.Exception);\n        return;\n    }\n\n    foreach (var msg in args.MessageList)\n    {\n        Console.WriteLine($\"Received data: {msg.GetDeserializedData()}\");\n        msg.Ack();\n    }\n};\n```\n\nThere may be some instances where you apply a schema *after* a station has received some messages. In order to consume those messages get_data_deserialized may be used to consume the messages without trying to apply the schema to them. As an example, if you produced a string to a station and then attached a protobuf schema, using get_data_deserialized will not try to deserialize the string as a protobuf-formatted message.\n\n### Fetch a single batch of messages\n\nUsing fetch_messages or fetch will allow the user to remove a specific number of messages from a given station. This behavior could be beneficial if the user does not want to have a consumer actively poll from a station indefinetly.\n\n```c#\nclient.FetchMessages(new FetchMessageOptions\n{\n    StationName= \"\u003cstation-name\u003e\",\n    ConsumerName= \"\u003cconsumer-name\u003e\",\n    ConsumerGroup= \"\u003cgroup-name\u003e\", // defaults to the consumer name.\n    BatchSize= 10, // defaults to 10\n    BatchMaxTimeToWaitMs= 1000, // defaults to 1000\n    MaxAckTimeMs= 30000, // defaults to 30000\n    MaxMsgDeliveries= 2, // defaults to 2\n    StartConsumeFromSequence= 1, // start consuming from a specific sequence. defaults to 1\n    LastMessages= -1 // consume the last N messages, defaults to -1 (all messages in the station)\n});\n```\n\n### Fetch a single batch of messages after creating a consumer\n\n`prefetch = true` will prefetch next batch of messages and save it in memory for future Fetch() request \\\nNote: Use a higher MaxAckTime as the messages will sit in a local cache for some time before processing\n\n```C#\nvar messages = consumer.Fetch(\n    batchSize: 10,\n    prefetch: true\n);\n```\n\n### Acknowledging a Message\n\nAcknowledging a message indicates to the Memphis server to not re-send the same message again to the same consumer or consumers group.\n\n```c#\nmsg.Ack();\n```\n\n### Nacking a Message\n\nMark the message as not acknowledged - the broker will resend the message immediately to the same consumers group, instead of waiting to the max ack time configured.\n\n```C#\nmsg.Nack();\n```\n\n### Sending a message to the dead-letter\n\nSending the message to the dead-letter station (DLS) - the broker won't resend the message again to the same consumers group and will place the message inside the dead-letter station (DLS) with the given reason.\nThe message will still be available to other consumer groups\n\n```shell\nmsg.DeadLetter(\"reason\");\n```\n\n### Delay the message after a given duration\n\nDelay the message and tell Memphis server to re-send the same message again to the same consumer group.\\\nThe message will be redelivered only in case `Consumer.MaxMsgDeliveries` is not reached yet.\n\n```C#\nmsg.Delay(\u003cdelayMilliSeconds\u003e);\n```\n\n### Get headers\n\nGet headers per message\n\n```c#\nmsg.GetHeaders()\n```\n\n### Get message sequence number \n\nGet message sequence number\n\n```C#\nvar sequence = msg.GetSequence();\n```\n\n### Get message time sent \n\nGet message time sent\n\n```C#\nvar dateTime = msg.GetTimeSent();\n```\n\n### Destroying a Consumer\n\n```c#\nawait consumer.DestroyAsync();\n```\n\n### Check if broker is connected\n\n```c#\nmemphisClient.IsConnected();\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsuperstreamlabs%2Fmemphis.net","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsuperstreamlabs%2Fmemphis.net","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsuperstreamlabs%2Fmemphis.net/lists"}