{"id":19438986,"url":"https://github.com/spacejam/rust-reactive-log","last_synced_at":"2025-07-15T09:07:24.297Z","repository":{"id":25239033,"uuid":"28663715","full_name":"spacejam/rust-reactive-log","owner":"spacejam","description":"performant transactional composable persistence","archived":false,"fork":false,"pushed_at":"2015-02-19T05:55:24.000Z","size":180,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-07-05T01:27:35.126Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/spacejam.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2014-12-31T10:44:34.000Z","updated_at":"2017-08-29T21:16:56.000Z","dependencies_parsed_at":"2022-08-23T22:00:48.481Z","dependency_job_id":null,"html_url":"https://github.com/spacejam/rust-reactive-log","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/spacejam/rust-reactive-log","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spacejam%2Frust-reactive-log","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spacejam%2Frust-reactive-log/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spacejam%2Frust-reactive-log/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spacejam%2Frust-reactive-log/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/spacejam","download_url":"https://codeload.github.com/spacejam/rust-reactive-log/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spacejam%2Frust-reactive-log/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265218751,"owners_count":23729530,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-10T15:20:31.901Z","updated_at":"2025-07-15T09:07:24.273Z","avatar_url":"https://github.com/spacejam.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"[WIP] rust-reactive-log\n=================\n\nThis readme is mostly the design spec at this point.  Most of this isn't implemented yet.\n\nLocal transactional log inspired by Acid-state and Kafka.  Delivery is strictly FIFO.\n\nConsumption styles:\n* per-consumer exactly once\n* global exactly once\n* nontransactional high-throughput (at least once)\n* all above styles expose both blocking and nonblocking interfaces\n\nConsumers can either use a simple iterator or a rich processing function that supports these macros:\n* ```commit!()``` completes processing of an element.  This is\n* ```retry!()``` restarts the processing attempt without incrementing the count of processed elements.\n* ```break!()``` causes the producer function to return.\n\nCaveats\n* only one process may produce at any time.  but feel free to create a proxy that funnels writes from multiple producers!\n* only one process may globally transactionally consume at a time\n* only one process may transactionally consume for a particular consumer ID at a time\n* none of these caveats are enforced by this library!\n\n```rust\nextern crate reactive-log;\nuse reactive-log::{Producer, Consumer, ProducerOptions, ConsumerOptions, SyncPolicy, Whence};\n\nfn producer() {\n  let prod_opts = ProducerOptions {\n      log_dir:                      \"/var/log/myapp/\",\n      sync_policy:                  SyncPolicy::Periodic(Duration::seconds(1)),\n      file_roll_size:               67_108_864,\n      blocking_minimum_retention:   None,\n      max_total_bytes:              536_870_912,\n      max_file_age:                 None,\n  };\n  let prod = Producer::new(prod_opts).unwrap();\n  ...\n  for data in input_stream.iter() {\n    prod.append(data.bytes());\n  }\n}\n\nfn simple_nontransactional_consumer() {\n  let whence = Whence::Latest;\n  let consumer_opts = ConsumerOptions {\n      log_dir:    \"/var/log/myapp/\",\n      style:      ConsumerStyle::NonTxConsumer(whence),\n  };\n\n  let consumer = Consumer::new(consumer_opts).unwrap();\n  for msg in consumer.iter() {\n    process(msg);\n  }\n}\n\nfn per_client_transactional_consumer() {\n  let consumer_id = \"spacejam's post consumer\";\n  let consumer_opts = ConsumerOptions {\n      log_dir:    \"/var/log/myapp/\",\n      style:      ConsumerStyle::ClientTxConsumer(consumer_id),\n  };\n\n  let consumer = Consumer::new(consumer_opts).unwrap();\n\n  // try to consume 5, but if we hit the end of the log don't wait to return (nonblocking + limit)\n  let max_messages_to_consume = Some(5);\n  consumer.nonblocking_process(max_messages_to_consume, |data| {\n      match process(data) {\n          Ok(processed) =\u003e match external_persist(processed) {\n              Ok(_) =\u003e commit!(),\n              Err(e) =\u003e {\n                  report(e);\n                  retry!();\n              },\n          },\n          Err(e) =\u003e {\n              report(e);\n              skip!();\n          },\n      }\n  })\n}\n\nfn global_transactional_consumer() {\n  let consumer_opts = ConsumerOptions {\n      log_dir:    \"/var/log/myapp/\",\n      style:      ConsumerStyle::GlobalTxConsumer(),\n  };\n\n  let consumer = Consumer::new(consumer_opts).unwrap();\n\n  // consume forever (blocking + no limit)\n  let max_messages_to_consume = None;\n  consumer.blocking_process(max_messages_to_consume, |data| {\n      match process(data) {\n          Ok(processed) =\u003e match external_persist(processed) {\n              Ok(_) =\u003e commit!(),\n              Err(e) =\u003e {\n                  report(e);\n                  retry!();\n              },\n          },\n          Err(e) =\u003e {\n              report(e);\n              skip!();\n          },\n      }\n  })\n}\n\nfn retrying_nontransactional_consumer() {\n  // subscribe to new messages, starting at the current last element\n  // position choices: Oldest, Latest, Position(offset)\n  let whence = Whence::Latest;\n  let consumer_opts = ConsumerOptions {\n      log_dir:    \"/var/log/myapp/\",\n      style:      ConsumerStyle::NonTxConsumer(whence),\n  };\n\n  let consumer = Consumer::new(consumer_opts).unwrap();\n\n  // consume forever (blocking + no limit)\n  let max_messages_to_consume = None;\n  consumer.blocking_process(max_messages_to_consume, |data| {\n      match process(data) {\n          Ok(processed) =\u003e match external_persist(processed) {\n              Ok(_) =\u003e commit!(),\n              Err(e) =\u003e {\n                  report(e);\n                  retry!();\n              },\n          },\n          Err(e) =\u003e {\n              report(e);\n              skip!();\n          },\n      }\n  })\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fspacejam%2Frust-reactive-log","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fspacejam%2Frust-reactive-log","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fspacejam%2Frust-reactive-log/lists"}