{"id":15405633,"url":"https://github.com/krisleech/conduit","last_synced_at":"2025-06-24T14:37:55.412Z","repository":{"id":24126183,"uuid":"27514874","full_name":"krisleech/conduit","owner":"krisleech","description":"An event store for Ruby","archived":false,"fork":false,"pushed_at":"2014-12-06T23:54:20.000Z","size":168,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-17T04:44:45.933Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/krisleech.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2014-12-04T00:20:13.000Z","updated_at":"2016-03-13T06:50:07.000Z","dependencies_parsed_at":"2022-08-22T11:10:37.223Z","dependency_job_id":null,"html_url":"https://github.com/krisleech/conduit","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/krisleech%2Fconduit","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/krisleech%2Fconduit/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/krisleech%2Fconduit/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/krisleech%2Fconduit/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/krisleech","download_url":"https://codeload.github.com/krisleech/conduit/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245359248,"owners_count":20602322,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-01T16:17:36.811Z","updated_at":"2025-03-24T21:41:07.900Z","avatar_url":"https://github.com/krisleech.png","language":"Ruby","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Conduit\n\nAn Event Store for Ruby.\n\nInstead of just storing the current state of an aggregate/entity in a data store we store an event, usually an action performed by a user. This provides a journel of how an aggregate/entity got to its current state.\n\nProjections create read-only denormalized views of your data meaning you can query your data without `JOIN`'s for optimal performance.\n\nIn essence reading and writing of data are seperate processes. Events are written and denormalized data is read. \n\nThe read data can, optionally, be constructed asynchronously on a different machine. If so the read data will become eventually consistent.\n\n## Installing\n\n```ruby\ngem 'conduit'\n```\n\n## Creating a store\n\n```ruby\n$store = Conduit::EventStore.new\n```\n\nBy default an in memory store is used for persistance, in production pass in a symbol, e.g. `:mongo`.\n\n## Pushing events\n\n```ruby\n$store.push(:thingy_created, 1, { ... })\n```\n\nEvents consist of an event name, the id of an aggregate/entity and a hash containing the details of the event. For example the hash might contain the changes to an aggregate/entity.\n\nNote that an id is required even for \"create\" events, as such the id must be supplied before any data is written to a data store, this negates the use of auto increment id's.\n\nTypically you will push an event from a domain service object, e.g.\n\n```ruby\nclass RegisterPerson  \n  class Form\n    include ActiveModel::Model\n    include Virtus\n    \n    attribute :first_name, String\n    attribute :last_name,  String\n    \n    validate :first_name, :last_name, presence: true\n  end\n  \n  def execute(form)\n    return false unless form.valid?\n        \n    id = SecureRandom.uuid    \n    $store.put(:person_registered, id, form.attributes)\n  end  \nend\n```\n\nEvents are always named in the past tense. It might seem odd to say something is done before it is even persisted to some permanent data store. But from the point of view of a user, as soon as they press a button, to them it is considered done, an action has been completed.\n\n## Reading a stream of events\n\n```ruby\nevents = $store.get(id)\n\nfirst_event = events[0]\n\nfirst_event.id # =\u003e ...\nfirst_event.name # =\u003e :person_registered\nfirst_event.data # =\u003e { first_name: 'Kris', last_name: 'Leech' }\nfirst_event.recorded_at # =\u003e ...\n```\n\nIt is up to you what you store in the `data` hash it might include details about who triggered the event.\n\nAn entity can be reconstructed from the events:\n\n```ruby\nclass Person\n  include Virtus\n  \n  attribute :id,         String\n  attribute :first_name, String\n  attribute :last_name,  String\nend\n\nclass ReplyPersonEvents\n  def call(events)\n    Person.new.tap |person|\n      events.each do |event| # TODO: replace with inject/reduce\n        person.attributes = data        \n      end  \n    end\n  end  \nend\n```\n\nIn most cases an entity will be made up of different kinds of events, e.g. \"person_registered\", \"person_updated\", \"person_retired\", which all reflect the domain language. If we needed to handle more than just \"person_registered\" in `ReplyPersonEvents` we could map each event to a different handler class.\n\n## Creating projections\n\nCreating aggregates/entities from an event stream will not be performant when the number of events increases.\n\nTo gain performance back we can create snapshots of the current state in a regular data store.\n\nNot only can we create the current state of the entity we can make it totally denormalized and create different denormalized views of the data, meaning no more slow `JOIN` or equivalent queries.\n\nWe can subscribe our own objects to the event store and they will be notified when an event occurs.\n\n```ruby\n$store.subscribe(PersonProjection.new)\n\nclass PersonProjection # TODO: better name, ProjectPerson?\n  def on_person_registered(event)\n    PersonRepo.__put__(data.merge(uuid: event.id))\n  end\nend\n```\n\n`PersonRepo` can be implimented in any way, to use any data store, for example a relation database could be implimented with `ActiveRecord`:\n\n```ruby\nclass PersonRepo \u003c ActiveRecord::Base  \n  set_table_name :people\n  \n  # this is only ever called from a projection\n  def self.__put__(attributes)\n    record = find_by_id(attributes.fetch(:uuid)) || new\n    record.update_attributes(attributes)\n  end\n  \n  def self.get(uuid)\n    record = find_by_id(uuid)\n    Person.new(record.attributes)\n  end\nend\n```\n\nFIXME: implimentation-wise it might be better to fetch the current entity from the repo and then use `ReplyPersonEvents` passing in the single event to get the current state and then save it. Otherwise we have two ways of setting state - reply and SQL.\n\nFIXME: should the repo be inside the projection and instead of \"getting\" from the repo we \"get\" from the projection? This would make sense when there are multiple projections as each would need its own repo.\n\nFIXME: or TDD it and see what happens...\n\nNow instead of replaying all the events we can grab the current state directly from a data store, pre-denormalized. \n\n```ruby\nperson = PersonRepo.get(uuid)\n```\n\nDisk space is cheap, instead of writing an finder method (SQL query) write a new projection passing in the historical events to get it up to date.\n\nProjections can be used to create temporary data views, e.g. for reporting purposes.\n\n### Async \n\nSubscribed projections can be configured to be called async using Celluloid or Sidekiq.\n\nIn a web app this would happen outside the request/response cycle and could be a problem if the new data is displayed stright away in the UI. If stale data is an issue and you still want async, you could recreate what will be eventually created in the data store.\n\n```ruby\n# add ability to reply on to an existing object\nclass ReplyPersonEvents\n  def call(events, options: {})\n    person = options.fetch(:onto, Person.new)\n    person.tap |person|\n      Array(events).each do |event| # TODO: replace with inject/reduce\n        person.attributes = data        \n      end  \n    end\n  end  \nend\n\n\nevent = $store.write(:person_registered, id, form.attributes)\n\n# we reply the single new event on to the existing entity\n@person = ReplyPersonEvents.call(event, onto: PersonRepo.fetch(params[:id])\n```\n\n# Things to consider\n\nWhat if the data store (SQL) is down or an error occurs? The snapshot can't be updated and will be stale/invalid. Maybe the store could require an `ack` from the subscriber and redeliver - but then we start to build an event bus, e.g. RabbitMQ. Or we could notify someone and allow them to fix it, events can be replayed from the date of last update to bring back up to date.\n\n```ruby\nPersonRepo.each do |record|\n  events = $store.read(record.uuid, since: record.updated_at)\n  person = ReplayPersonEvents.call(events)\n  record.update(person.attributes)\nend\n```\n\nThis could even happen continually in a seperate process, in which case there is no need to subscribe any projections (but it might be better for development/debugging purposes).\n\n## Development\n\n```\nminispec\n```\n\n## Contributing\n\n1. Fork it ( https://github.com/[my-github-username]/conduit/fork )\n2. Create your feature branch (`git checkout -b my-new-feature`)\n3. Commit your changes (`git commit -am 'Add some feature'`)\n4. Push to the branch (`git push origin my-new-feature`)\n5. Create a new Pull Request\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkrisleech%2Fconduit","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkrisleech%2Fconduit","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkrisleech%2Fconduit/lists"}