{"id":18073473,"url":"https://github.com/iamazy/krost","last_synced_at":"2025-04-05T18:17:30.630Z","repository":{"id":101767169,"uuid":"488479327","full_name":"iamazy/krost","owner":"iamazy","description":"an automated mechanism for serializing structured data in kafka protocol.","archived":false,"fork":false,"pushed_at":"2022-05-08T10:46:01.000Z","size":218,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-11T15:33:12.299Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/iamazy.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-05-04T06:40:05.000Z","updated_at":"2022-05-04T06:40:13.000Z","dependencies_parsed_at":"2023-07-17T02:15:48.862Z","dependency_job_id":null,"html_url":"https://github.com/iamazy/krost","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iamazy%2Fkrost","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iamazy%2Fkrost/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iamazy%2Fkrost/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iamazy%2Fkrost/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/iamazy","download_url":"https://codeload.github.com/iamazy/krost/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247378144,"owners_count":20929296,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-31T10:07:45.322Z","updated_at":"2025-04-05T18:17:30.616Z","avatar_url":"https://github.com/iamazy.png","language":"Rust","readme":"# krost\n\nkrost is a flexible, efficient, automated mechanism for serializing structured data in kafka protocol. Mainly affected by [Prost](https://github.com/tokio-rs/prost) (a protobuf implementation for rust).\n\nThis crate still have a lot of work to do, but it's a good start.\n\n### Codegen example\n\n`ProduceRequest` definition: \n\n```json\n{\n  \"apiKey\": 0,\n  \"type\": \"request\",\n  \"listeners\": [\"zkBroker\", \"broker\"],\n  \"name\": \"ProduceRequest\",\n  // Version 1 and 2 are the same as version 0.\n  //\n  // Version 3 adds the transactional ID, which is used for authorization when attempting to write\n  // transactional data.  Version 3 also adds support for Kafka Message Format v2.\n  //\n  // Version 4 is the same as version 3, but the requestor must be prepared to handle a\n  // KAFKA_STORAGE_ERROR. \n  //\n  // Version 5 and 6 are the same as version 3.\n  //\n  // Starting in version 7, records can be produced using ZStandard compression.  See KIP-110.\n  //\n  // Starting in Version 8, response has RecordErrors and ErrorMEssage. See KIP-467.\n  //\n  // Version 9 enables flexible versions.\n  \"validVersions\": \"0-9\",\n  \"flexibleVersions\": \"9+\",\n  \"fields\": [\n    { \"name\": \"TransactionalId\", \"type\": \"string\", \"versions\": \"3+\", \"nullableVersions\": \"3+\", \"default\": \"null\", \"entityType\": \"transactionalId\",\n      \"about\": \"The transactional ID, or null if the producer is not transactional.\" },\n    { \"name\": \"Acks\", \"type\": \"int16\", \"versions\": \"0+\",\n      \"about\": \"The number of acknowledgments the producer requires the leader to have received before considering a request complete. Allowed values: 0 for no acknowledgments, 1 for only the leader and -1 for the full ISR.\" },\n    { \"name\": \"TimeoutMs\", \"type\": \"int32\", \"versions\": \"0+\",\n      \"about\": \"The timeout to await a response in milliseconds.\" },\n    { \"name\": \"TopicData\", \"type\": \"[]TopicProduceData\", \"versions\": \"0+\",\n      \"about\": \"Each topic to produce to.\", \"fields\": [\n      { \"name\": \"Name\", \"type\": \"string\", \"versions\": \"0+\", \"entityType\": \"topicName\", \"mapKey\": true,\n        \"about\": \"The topic name.\" },\n      { \"name\": \"PartitionData\", \"type\": \"[]PartitionProduceData\", \"versions\": \"0+\",\n        \"about\": \"Each partition to produce to.\", \"fields\": [\n        { \"name\": \"Index\", \"type\": \"int32\", \"versions\": \"0+\",\n          \"about\": \"The partition index.\" },\n        { \"name\": \"Records\", \"type\": \"records\", \"versions\": \"0+\", \"nullableVersions\": \"0+\",\n          \"about\": \"The record data to be produced.\" }\n      ]}\n    ]}\n  ]\n}\n```\n\nOutput:\n\n```rust\npub mod request {\n    pub mod produce {\n        #[derive(Debug, PartialEq, krost_derive::Message, Clone)]\n        #[kafka(apikey = 0i16, versions = \"0-9\", flexible = \"9+\")]\n        pub struct ProduceRequest {\n            ///The transactional ID, or null if the producer is not transactional.\n            #[kafka(versions = \"3+\", nullable = \"3+\", default = \"null\")]\n            pub transactional_id: Option\u003cString\u003e,\n            ///The number of acknowledgments the producer requires the leader to have received before considering a request complete. Allowed values: 0 for no acknowledgments, 1 for only the leader and -1 for the full ISR.\n            #[kafka(versions = \"0+\")]\n            pub acks: i16,\n            ///The timeout to await a response in milliseconds.\n            #[kafka(versions = \"0+\")]\n            pub timeout_ms: i32,\n            ///Each topic to produce to.\n            #[kafka(versions = \"0+\")]\n            pub topic_data: Vec\u003cTopicProduceData\u003e,\n            ///The tagged fields.\n            #[kafka(versions = \"9+\")]\n            pub _tagged_fields: krost::types::TaggedFields,\n        }\n        #[derive(Debug, PartialEq, krost_derive::Message, Clone)]\n        pub struct PartitionProduceData {\n            ///The partition index.\n            #[kafka(versions = \"0+\")]\n            pub index: i32,\n            ///The record data to be produced.\n            #[kafka(versions = \"0+\", nullable = \"0+\")]\n            pub records: Option\u003ckrost::record::RecordBatch\u003e,\n            ///The tagged fields.\n            #[kafka(versions = \"9+\")]\n            pub _tagged_fields: krost::types::TaggedFields,\n        }\n        #[derive(Debug, PartialEq, krost_derive::Message, Clone)]\n        pub struct TopicProduceData {\n            ///The topic name.\n            #[kafka(versions = \"0+\")]\n            pub name: String,\n            ///Each partition to produce to.\n            #[kafka(versions = \"0+\")]\n            pub partition_data: Vec\u003cPartitionProduceData\u003e,\n            ///The tagged fields.\n            #[kafka(versions = \"9+\")]\n            pub _tagged_fields: krost::types::TaggedFields,\n        }\n    }\n}\n```\n\nand you could find whole code in `krost/tests/krost.rs`\n\n### Acknowledgments\n\n- [tychedelia/kafka-protocol-rs](https://github.com/tychedelia/kafka-protocol-rs)\n- [gardnervickers/kafka-protocol-rs](https://github.com/gardnervickers/kafka-protocol-rs)\n- [influxdata/rskafka](https://github.com/influxdata/rskafka)\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fiamazy%2Fkrost","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fiamazy%2Fkrost","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fiamazy%2Fkrost/lists"}