{"id":20945235,"url":"https://github.com/jyotindersingh/eventide-ci","last_synced_at":"2025-08-16T12:07:46.808Z","repository":{"id":193858364,"uuid":"404452005","full_name":"JyotinderSingh/eventide-ci","owner":"JyotinderSingh","description":"A distributed Continuous Integration System","archived":false,"fork":false,"pushed_at":"2021-09-11T12:28:10.000Z","size":29,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-06-01T08:33:55.714Z","etag":null,"topics":["continuous-integration","distributed-systems","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/JyotinderSingh.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2021-09-08T18:20:13.000Z","updated_at":"2021-09-11T12:28:12.000Z","dependencies_parsed_at":"2023-09-10T13:50:43.089Z","dependency_job_id":null,"html_url":"https://github.com/JyotinderSingh/eventide-ci","commit_stats":null,"previous_names":["jyotindersingh/eventide-ci"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/JyotinderSingh/eventide-ci","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JyotinderSingh%2Feventide-ci","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JyotinderSingh%2Feventide-ci/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JyotinderSingh%2Feventide-ci/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JyotinderSingh%2Feventide-ci/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/JyotinderSingh","download_url":"https://codeload.github.com/JyotinderSingh/eventide-ci/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JyotinderSingh%2Feventide-ci/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":270709019,"owners_count":24631992,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-16T02:00:11.002Z","response_time":91,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["continuous-integration","distributed-systems","python"],"created_at":"2024-11-18T23:47:14.087Z","updated_at":"2025-08-16T12:07:46.758Z","avatar_url":"https://github.com/JyotinderSingh.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🌘 Eventide \n\n### Eventide is a simple distributed Continuous Integration system.\n\nThis is an **initial prototype** for a distributed Continuous Integration System.\n\n---\n\n## Features\n\n- Distributed architecture for scalability - supports virtually unlimited concurrent test runners.\n- New test runners auto register themselves with the cluster.\n- Test results get stored in a file with a summary of the test runs.\n- Includes heartbeat mechanism among different services, which allows for some simple error handling:\n  - If a test runner dies, the dispatcher service will automatically figure this out and remove it from the pool - and will give another runner (if available) the job. Otherwise, it will wait for the current test runners to finish processing or a new test runner to register itself in the pool.\n  - If the dispatcher service dies, the test runners and repo observer service will notice this and gracefully shut down.\n\n## Architecture\n\nThe system is built of 3 main components:\n\n- **Observer Service**: This service monitors a git repository for new changes - when a commit is detected it notifies the dispatcher service about the same.\n- **Dispatcher Service**: Responsible for distributing incoming jobs (commit IDs against which we need to run a test) to available test runners. Also reponsible for registering new test runners that try to register themselves with the cluster. In case a runner crashes, the dispatcher service removes it from the registry. In case a test runner fails while running a test, the test a re-assigned to another healthy node.\n- **Test Runner Service**: Responsible for actually executing the tests on the given commit IDs. Ideally has a large number of nodes to allow several concurrent test runs.\n\n## Steps to run\n\n### Setup\n\nCreate a repository you want Eventide to monitor. Currently Eventide supports git repositories only.\n\nThis is the main repository where the developers will check in their code.\n\n```\nmkdir test_repo\ncd test_repo\ngit init\n```\n\nTo get started, we need at least one commit in this master repository. You can add the [example tests provided here](./tests) for testing purposes.\n\nYou would want your tests to be present in a directory called `tests`. This is where Eventide looks for test files to be run.\n\n```\ncp -r /this/directory/tests/ /path/to/test_repo/\ncd /path/to/test_repo\ngit add tests/\ngit commit -m \"add tests\"\n```\n\nThe Repo Observer service will need its own copy of the repository, since it might exist on a different system altogether.\nLet's create a clone of the master repo\n\n```\ngit clone /path/to/test_repo test_repo_clone_observer\n```\n\nSimilarly, the Test Runner service will also need its own copy of the repo\n\n```\ngit clone /path/to/test_repo test_repo_clone_runner\n```\n\n### Running the CI\n\nWe will be using three different terminal shells to run this distributed system. You can run these on different machines as well - however you will need to provide additional arguments for the host and port numbers to the scripts.\n\nWe start the dispatcher service first, it listens for dispatch commands from the Observer service and launches tests on the Test Runner service.\n\n```\npython dispatcher_srv.py\n```\n\nIn a new shell we start the Test Runner service - which authomatically registers itself with the dispatcher. You can start as many test runners as you'd like. Multiple test runners will provide better concurrent performance.\n\n```\npython test_runner_srv.py \u003cpath/to/test_repo_clone_runner\u003e\n```\n\nThe test runner will assign itself its own port in the range 8900-9000\n\nFinally, we start the Repo Observer service.\n\n```\npython repo_observer_srv.py --dispatcher-server=localhost:8888 \u003cpath/to/test_repo_clone_observer\u003e\n```\n\nNow Eventide is ready to run some tests!\n\n### Launching an automated test\n\nTo launch a new run, we need to make a new commit.\n\nGo to the master repository, and make an arbitrary change:\n\n```\ncd /path/to/test_repo\ntouch new_file\ngit add new_file\ngit commit -m \"new file\"\n```\n\nThe `repo_observer_srv.py` will realize there is a new commit and notify the dispatcher service. You will be able to see the output in their respective shells and monitor the progress.\n\nOnce the dispatcher receives the results, it stores them in a `test_results/` folder in the code base, using the `commit ID` as the filename.\n\n## Configuring the services\n\nYou can add the following command line arguments when launching the services.\n\n### Repo Observer Service\n\n- `--dispatcher-server`\n\n  Dispatcher `host:port`, by default it uses `localhost:8888`\n\n- `repo`\n\n  path to the repository to be observed (needs a specific clone for itself)\n\n### Dispatcher Service\n\n- `--host`\n\n  Dispatcher service's host, by default it uses `localhost`\n\n- `--port`\n\n  Port on which the service will listen, by default it uses `8888`\n\n### Test Runner Service\n\n- `--host`\n\n  Host for the test runner instance, by default it uses `localhost`\n\n- `--port`\n\n  Port on which the test runner will listen, by default it uses values \u003e= `8900`\n\n- `--dispatcher-server`\n\n  Dispatcher Service's `host:port`, by default it uses `localhost:8888`\n\n- `repo`\n\n  Path to the repository this will observe.\n\n## Limitations\n\nThis software is currently under development, and has a few limitations:\n\n- Only local git repositories are supported as of now.\n- The test runner only checks the `tests/` directory in the codebase to look for tests to run. This is currently not configurable.\n- Eventide checks for changes to the repo every 5 seconds. It does not test every commit made in these 5 seconds, only the most recent commit.\n- The test results are stored on the file system local to the Dispatcher service. A better approach would be to send this to another service that hosts these results on the web for easier access.\n\n## Development in progess\n\nThe current architecture is an initial proof of concept.\n\nThe final product expects to use a robust event queue for dispatching events among services, and support high availablity of the services - along with easier configuration management. We would also want to support more robust hook based workflows to detect changes in the codebase.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjyotindersingh%2Feventide-ci","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjyotindersingh%2Feventide-ci","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjyotindersingh%2Feventide-ci/lists"}