{"id":15667938,"url":"https://github.com/rupurt/branch","last_synced_at":"2025-03-30T04:41:32.029Z","repository":{"id":215755100,"uuid":"739709499","full_name":"rupurt/branch","owner":"rupurt","description":"A streaming toolkit with bottomless storage","archived":false,"fork":false,"pushed_at":"2024-01-18T00:13:16.000Z","size":30,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-02-05T06:46:22.982Z","etag":null,"topics":["azure-blob-storage","gcs","kafka","kappa","kappa-architecture","minio","object-storage","python","r2","s3","streaming"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rupurt.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-01-06T09:56:30.000Z","updated_at":"2024-01-30T06:10:27.000Z","dependencies_parsed_at":"2024-01-08T22:27:10.193Z","dependency_job_id":"59abadcf-96d8-4b92-b9fa-4ea27b4cc258","html_url":"https://github.com/rupurt/branch","commit_stats":{"total_commits":1,"total_committers":1,"mean_commits":1.0,"dds":0.0,"last_synced_commit":"fe99fc44f8c7cd0be792356d5dc852c6179001da"},"previous_names":["rupurt/branch"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rupurt%2Fbranch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rupurt%2Fbranch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rupurt%2Fbranch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rupurt%2Fbranch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rupurt","download_url":"https://codeload.github.com/rupurt/branch/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246277351,"owners_count":20751548,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["azure-blob-storage","gcs","kafka","kappa","kappa-architecture","minio","object-storage","python","r2","s3","streaming"],"created_at":"2024-10-03T14:05:53.119Z","updated_at":"2025-03-30T04:41:31.997Z","avatar_url":"https://github.com/rupurt.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Branch\n\nA streaming toolkit with bottomless storage\n\n- [x] Embedded\n- [ ] Single node\n- [ ] Clustered\n- [x] HTTP API\n- [ ] Web UI\n- [ ] Kafka gateway\n- [ ] S3\n- [x] GCS\n- [ ] Azure blob storage\n- [ ] MinIO\n- [ ] R2\n- [x] Local\n- [x] Python bindings\n- [ ] Elixir bindings\n- [ ] Zig bindings\n- [ ] C bindings\n\n## Usage\n\n### CLI\n\n```shell\n\u003e branch --help\n\n Usage: branch [OPTIONS] COMMAND [ARGS]...\n\n╭─ Options ──────────────────────────────────────────────────────────────────────────────────────────────────────╮\n│ --install-completion          Install completion for the current shell.                                        │\n│ --show-completion             Show completion for the current shell, to copy it or customize the installation. │\n│ --help                        Show this message and exit.                                                      │\n╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯\n╭─ Commands ─────────────────────────────────────────────────────────────────────────────────────────────────────╮\n│ init                                                                                                         │\n│ produce                                                                                                        │\n│ consume                                                                                                        │\n│ topics                                                                                                         │\n│ server                                                                                                         │\n╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯\n```\n\nCreate your configuration file by following the prompts\n\n```shell\n\u003e branch init\n...\n```\n\nCreate a topic\n\n```shell\n\u003e branch topics create \\\n    --topic=my-topic.v1 \\\n    --partitions=3\n```\n\nProduce a new record every second\n\n```shell\n\u003e watch -n 1 \\\n    'branch produce --topic=my-topic.v1 --partition=0 --message=\\'{\"hello\": \"cli!\", \"now\": \"$(date --iso-8601=ns)\"}\\''\n...\n```\n\nConsume records produced to `my-topic.v1`\n\n```shell\n\u003e branch consume --topic=my-topic.v1\n...\n```\n\n### Python\n\n```python\n# my-branch.py\nimport branch\nfrom datetime import datetime\n\nmsg = '{\"hello\": \"python!\", \"now\": datetime.now()}'\nproducer_record = branch.ProducerRecord(topic=\"my-topic.v1\", value=msg)\nbranch.produce(producer_record)\nbranch.consume(\"my-topic.v1\")\n```\n\n```shell\n\u003e python my-branch.py\n...\n```\n\n## Single node\n\n```shell\n\u003e branch server start --port=9000 --ui-port=8999\n...\n```\n\n## Clustered\n\n```shell\n\u003e branch server start --host=0.0.0.0 --port=9001 --name=my-branch-1\n...\n\u003e branch server start --host=0.0.0.0 --port=9002 --name=my-branch-2\n...\n```\n\n## HTTP\n\n```shell\n\u003e curl -X POST -d '{...}' http://localhost:9000/api/v1/produce\n...\n```\n\n```shell\n\u003e curl http://localhost:9000/api/v1/consume\n...\n```\n\n## Web UI\n\n```shell\n\u003e open http://localhost:8999\n```\n\n## Development\n\nThis repository manages the dev environment as a [Nix flake](https://nixos.wiki/wiki/Flakes)\nand requires [Nix to be installed](https://github.com/DeterminateSystems/nix-installer)\n\n```shell\nnix develop -c $SHELL\n```\n\n```shell\nmake all\n```\n\n```shell\nmake deps.install\nmake deps.install/dev\n```\n\n```shell\nmake deps.freeze\nmake deps.freeze/dev\n```\n\n```shell\nmake deps/reqs\nmake deps/outdated\nmake deps/check\n```\n\n```shell\nmake run\nmake run/server\n```\n\n```shell\nmake format\nmake format/check\n```\n\n```shell\nmake typecheck\n```\n\n```shell\nmake lint\n```\n\n```shell\nmake test\nmake test/watch\n```\n\n```shell\nmake coverage\n```\n\n```shell\nmake build\nmake build/pyinstaller\nmake build/nuitka\n```\n\n```shell\nmake exec\n```\n\n```shell\nmake clean\n```\n\n```shell\nmake compose.build\nmake compose.up\n```\n\n```shell\nmake docker.build/deps\nmake docker.build/run\n```\n\n## License\n\n`branch` is released under the [MIT license](./LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frupurt%2Fbranch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frupurt%2Fbranch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frupurt%2Fbranch/lists"}