{"id":27993954,"url":"https://github.com/erleans/vonnegut","last_synced_at":"2026-03-11T10:32:39.142Z","repository":{"id":66571887,"uuid":"45929867","full_name":"erleans/vonnegut","owner":"erleans","description":null,"archived":false,"fork":false,"pushed_at":"2018-08-15T22:47:24.000Z","size":515,"stargazers_count":61,"open_issues_count":12,"forks_count":6,"subscribers_count":13,"default_branch":"master","last_synced_at":"2025-10-11T03:35:23.837Z","etag":null,"topics":["distributed-log","erlang","kafka"],"latest_commit_sha":null,"homepage":null,"language":"Erlang","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/erleans.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2015-11-10T17:53:02.000Z","updated_at":"2025-02-08T05:48:26.000Z","dependencies_parsed_at":"2023-06-01T19:30:29.907Z","dependency_job_id":null,"html_url":"https://github.com/erleans/vonnegut","commit_stats":null,"previous_names":[],"tags_count":21,"template":false,"template_full_name":null,"purl":"pkg:github/erleans/vonnegut","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/erleans%2Fvonnegut","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/erleans%2Fvonnegut/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/erleans%2Fvonnegut/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/erleans%2Fvonnegut/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/erleans","download_url":"https://codeload.github.com/erleans/vonnegut/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/erleans%2Fvonnegut/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30378091,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-11T06:09:32.197Z","status":"ssl_error","status_checked_at":"2026-03-11T06:09:17.086Z","response_time":84,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["distributed-log","erlang","kafka"],"created_at":"2025-05-08T19:04:24.653Z","updated_at":"2026-03-11T10:32:39.126Z","avatar_url":"https://github.com/erleans.png","language":"Erlang","funding_links":[],"categories":[],"sub_categories":[],"readme":"vonnegut\n=====\n\n[![CircleCI](https://circleci.com/gh/SpaceTime-IoT/vonnegut.svg?style=svg)](https://circleci.com/gh/SpaceTime-IoT/vonnegut)\n\n[![codecov](https://codecov.io/gh/SpaceTime-IoT/vonnegut/branch/master/graph/badge.svg)](https://codecov.io/gh/SpaceTime-IoT/vonnegut)\n\nVonnegut is a append-only log that follows the file format and API of Kafka 1.0. The server can be run standalone, with 1 or more chains each with 1 or more replicas, or as part of another Erlang release which can talk to it directly.\n\nEach chain is responsible for a range of the topic space. A read or write to a topic requires finding what chain the topic belongs to and then making a request to the head, in the case of a write, or the tail, in the case of a read.\n\nConfiguration\n-----\n\n### Server\n\nA node in a chain can discover other nodes within the chain through DNS SRV record queries. The `replicas` configuration tells vonnegut node how many other nodes it needs to connect to to form the required chain length to ack writes.\n\n```\n{vonnegut, [{chain, [{name, \"chain-1\"},\n                     {discovery, {srv, \"chain-1.service.cluster.local\"}},\n                     {replicas, \"2\"},\n                     {port, 5555}]}\n            ]}\n```\n\n### Client\n\nClients start a pool of connections to the head and tail of each chain. Chains are found through DNS queries against endpoints:\n\n```\n{vonnegut, [{client, [{endpoints, [{\"chain-1.service.cluster.local\", 5555}]}]}]}\n```\n\nErlang Interface\n---\n\nA local interface can be used to create, read and write topics. \n\n```shell\n$ rebar3 shell\n1\u003e vg:create_topic(\u003c\u003c\"test_topic\"\u003e\u003e).\n2\u003e vg:write(\u003c\u003c\"test_topic\"\u003e\u003e, [\u003c\u003c\"some log message\"\u003e\u003e, \u003c\u003c\"more log message\"\u003e\u003e]).\n3\u003e vg:fetch(\u003c\u003c\"test_topic\"\u003e\u003e).\n{ok,#{high_water_mark =\u003e 1,partition =\u003e 0,\n      record_batches =\u003e\n          [#{headers =\u003e [],key =\u003e \u003c\u003c\u003e\u003e,offset =\u003e 1,sequence_number =\u003e 1,\n             timestamp =\u003e 1517613646458,value =\u003e \u003c\u003c\"more log message\"\u003e\u003e},\n           #{headers =\u003e [],key =\u003e \u003c\u003c\u003e\u003e,offset =\u003e 0,sequence_number =\u003e 0,\n             timestamp =\u003e 1517613646458,\n             value =\u003e \u003c\u003c\"some log message\"\u003e\u003e}]}}\n```\n\nBy default index and log files will be written to `./data`:\n\n```shell\n$ ls data/test_topic-0/\n00000000000000000000.index  00000000000000000000.log\n```\n\nKafkaesque Client\n---\n\n```erlang\n$ rebar3 shell\n1\u003e vg_client_pool:start().\nok\n2\u003e vg_client:produce(\u003c\u003c\"my-topic-2\"\u003e\u003e, [\u003c\u003c\"message 1\"\u003e\u003e, \u003c\u003c\"message 2\"\u003e\u003e]).\n{ok,1}\n3\u003e vg_client:fetch(\u003c\u003c\"my-topic-2\"\u003e\u003e).\n{ok,#{\u003c\u003c\"test_topic-2\"\u003e\u003e =\u003e\n          #{0 =\u003e\n                #{error_code =\u003e 0,high_water_mark =\u003e 1,\n                  record_batches =\u003e\n                      [#{headers =\u003e [],key =\u003e \u003c\u003c\u003e\u003e,offset =\u003e 1,\n                         sequence_number =\u003e 1,timestamp =\u003e 1517616861441,\n                         value =\u003e \u003c\u003c\"message 2\"\u003e\u003e},\n                       #{headers =\u003e [],key =\u003e \u003c\u003c\u003e\u003e,offset =\u003e 0,\n                         sequence_number =\u003e 0,timestamp =\u003e 1517616861441,\n                         value =\u003e \u003c\u003c\"message 1\"\u003e\u003e}],\n                  record_batches_size =\u003e 95}}}}\n```\n\nRunning Tests\n-----\n\nThe tests require opening thousands of files and so may require increasing the limit per process on your system with:\n\n```shell\n$ ulimit -n 63536\n```\n\nTests also require a nodename:\n\n```shell\n$ rebar3 ct\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ferleans%2Fvonnegut","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ferleans%2Fvonnegut","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ferleans%2Fvonnegut/lists"}