{"id":26412999,"url":"https://github.com/arjun-1/kafkagosaur","last_synced_at":"2025-03-17T22:53:45.903Z","repository":{"id":43366036,"uuid":"435230953","full_name":"arjun-1/kafkagosaur","owner":"arjun-1","description":"Kafka client for Deno binding to kafka-go using WebAssembly","archived":false,"fork":false,"pushed_at":"2023-02-25T00:13:59.000Z","size":13989,"stargazers_count":15,"open_issues_count":5,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-02-17T06:17:43.814Z","etag":null,"topics":["deno","golang","kafka","typescript","wasm","webassembly"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/arjun-1.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"security/sasl.ts","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-12-05T17:18:04.000Z","updated_at":"2024-12-17T03:45:01.000Z","dependencies_parsed_at":"2024-06-21T14:10:44.882Z","dependency_job_id":"ef43ac15-cfc8-495f-a3db-b057a6d995c4","html_url":"https://github.com/arjun-1/kafkagosaur","commit_stats":{"total_commits":48,"total_committers":1,"mean_commits":48.0,"dds":0.0,"last_synced_commit":"ef469da1445239a4ad02a3742db94025e9363514"},"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arjun-1%2Fkafkagosaur","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arjun-1%2Fkafkagosaur/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arjun-1%2Fkafkagosaur/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arjun-1%2Fkafkagosaur/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/arjun-1","download_url":"https://codeload.github.com/arjun-1/kafkagosaur/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244042160,"owners_count":20388360,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deno","golang","kafka","typescript","wasm","webassembly"],"created_at":"2025-03-17T22:53:45.397Z","updated_at":"2025-03-17T22:53:45.892Z","avatar_url":"https://github.com/arjun-1.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# kafkagosaur\n\n[![deno module](https://img.shields.io/endpoint?url=https%3A%2F%2Fdeno-visualizer.danopia.net%2Fshields%2Flatest-version%2Fx%2Fkafkagosaur%2Fmod.ts)](https://deno.land/x/kafkagosaur)\n[![deno doc](https://doc.deno.land/badge.svg)](https://doc.deno.land/https/deno.land/x/kafkagosaur/mod.ts)\n[![CI](https://github.com/arjun-1/kafkagosaur/actions/workflows/CI.yaml/badge.svg)](https://github.com/arjun-1/kafkagosaur/actions/workflows/CI.yaml)\n[![codecov](https://codecov.io/gh/arjun-1/kafkagosaur/branch/master/graph/badge.svg)](https://codecov.io/gh/arjun-1/kafkagosaur)\n\nKafkagosaur is a Kafka client for Deno built using WebAssembly. The project\nbinds to the [kafka-go](https://github.com/segmentio/kafka-go) library.\n\nSee [this article](https://medium.com/@arjun.dhawan/kafkagosaur-eac3c063388) for\nan overview of its inner workings.\n\n## Supported features\n\n- [x] Writer\n- [x] Reader\n- [x] SASL\n- [x] TLS\n- [ ] Deno streams\n\n## Examples\n\nFor comprehensive examples on how to use kafkagosaur, head over to the\n[provided examples](#provided-examples).\n\n#### KafkaWriter\n\nTo write a message, first create `KafkaWriter` instance using the `createWriter`\nfunction on the `KafkaGoSaur` instance:\n\n```typescript\nimport KafkaGoSaur from \"https://deno.land/x/kafkagosaur/mod.ts\";\n\nconst kafkaGoSaur = new KafkaGoSaur();\nconst writer = await kafkaGoSaur.createWriter({\n  broker: \"localhost:9092\",\n  topic: \"test-0\",\n});\n\nconst enc = new TextEncoder();\nconst msgs = [{ value: enc.encode(\"value\") }];\n\nawait writer.writeMessages(msgs);\n```\n\n#### KafkaReader\n\nTo read a message, first create `KafkaReader` instance using the `createReader`\nfunction on the `KafkaGoSaur` instance:\n\n```typescript\nimport KafkaGoSaur from \"https://deno.land/x/kafkagosaur/mod.ts\";\n\nconst kafkaGoSaur = new KafkaGoSaur();\nconst reader = await kafkaGoSaur.createReader({\n  brokers: [\"localhost:9092\"],\n  topic: \"test-0\",\n});\n\nconst readMsg = await reader.readMessage();\n```\n\n### Provided examples\n\nTo run the [provided examples](examples), ensure you have docker up and running.\nThen start the kafka broker using\n\n```bash\nmake docker-up\n```\n\nTo run the writer example\n\n```bash\ndeno run --allow-read --allow-net --unstable examples/writer.ts\n```\n\nTo run the reader example\n\n```bash\ndeno run --allow-read --allow-net --unstable examples/reader.ts\n```\n\n## Documentation\n\nThe API documentation is hosted\n[here](https://doc.deno.land/https/deno.land/x/kafkagosaur/mod.ts).\n\n## Development\n\nTo build the WebAssembly module, first run\n\n```bash\nmake build-wasm\n```\n\nTo run the tests, ensure first you have docker up and running. Then start the\nkafka broker using\n\n```bash\nmake docker-up\n```\n\nThen run\n\n```bash\nmake test\n```\n\n## Performance benchmarks\n\nThe Deno benchmarks are located in [bench](bench) and can be run via\n\n```bash\ndeno run --allow-read --allow-net --allow-env --unstable bench/reader.ts\n```\n\n```bash\ndeno run --allow-read --allow-net --allow-env --unstable bench/writer.ts\n```\n\n### Results\n\n\u003cimg width=\"486\" alt=\"kafkagosaur-perf\" src=\"https://user-images.githubusercontent.com/8102654/156884252-e9d7b735-7d3d-4162-a3b3-2854a1ae9baf.png\"\u003e\n\n|                   | kafka-go[^2] | kafkagosaur (`DialBackend.Node`) | kafkagosaur (`DialBackend.Deno`) |\n| ----------------- | ------------ | -------------------------------- | -------------------------------- |\n| fetchMessage      | 5236 ± 743   | 2760 ± 376                       | 2678 ± 361                       |\n| writeMessages[^1] | 5333 ± 148   | 4455 ± 159                       | N/A[^3]                          |\n\n[^1]: Batching 10.000 messages.\n\n[^2]: Using a single goroutine.\n\n[^3]: `DialBackend.Deno` not yet functional for writing.\n\n#### Environment\n\n- 2,6 GHz 6-Core Intel Core i7\n- [Confluent Cloud Basic cluster](https://docs.confluent.io/cloud/current/clusters/cluster-types.html#basic-clusters);\n  6 partitions\n\n## Contributing\n\nKafkgagosaur is in early stage of development. Nevertheless your contributions\nare highly valued and welcomed! Feel free to ask for new features, report bugs,\nor submit your code.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Farjun-1%2Fkafkagosaur","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Farjun-1%2Fkafkagosaur","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Farjun-1%2Fkafkagosaur/lists"}