{"id":22169809,"url":"https://github.com/akopdev/callisto","last_synced_at":"2025-03-24T17:24:01.717Z","repository":{"id":261422050,"uuid":"884248520","full_name":"akopdev/callisto","owner":"akopdev","description":"Perform machine learning experiments without jupyter notebooks","archived":false,"fork":false,"pushed_at":"2024-11-25T15:44:55.000Z","size":13,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-29T22:18:21.925Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/akopdev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-11-06T12:17:19.000Z","updated_at":"2024-11-25T15:45:10.000Z","dependencies_parsed_at":"2024-11-06T13:54:24.817Z","dependency_job_id":"ecf6737f-c398-483b-b1e3-d72a39ec47a8","html_url":"https://github.com/akopdev/callisto","commit_stats":null,"previous_names":["akopdev/callisto"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/akopdev%2Fcallisto","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/akopdev%2Fcallisto/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/akopdev%2Fcallisto/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/akopdev%2Fcallisto/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/akopdev","download_url":"https://codeload.github.com/akopdev/callisto/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245316282,"owners_count":20595410,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-02T06:36:01.775Z","updated_at":"2025-03-24T17:24:01.693Z","avatar_url":"https://github.com/akopdev.png","language":"Python","readme":"Callisto\n========\n\nMicro-framework, to perform machine learning experiments without jupyter notebooks.\n\nThe main goal of Callisto is to provide a simple task orchestrator to manage the execution of data pipelines and \neffectively manage results. Just like a jupyter notebook allows you to execute only parts of the code, \nCallisto will cache unchanged tasks, and execute only necessary steps, allowing you to run pipelines multiple \ntimes with the expected results.\n\nYou can also use Callisto to manage cross task data, by requesting already computed output of one task \nas an input for another.\n\n\n## Installation\n\n```bash\npip install git+https://github.com/akopdev/callisto.git\n```\n\n## Usage example\n\nLet's create a simple pipeline that will filter data from a list of dictionaries.\n\n```python\nfrom callisto import Callisto\n\napp = Callisto()\n\n@app.task(name=\"step1\")\ndef get_data():\n    return [\n        {\"id\": 1, \"name\": \"John\", \"last_name\": \"Doe\", \"age\": 30},\n        {\"id\": 2, \"name\": \"Jane\", \"last_name\": \"Doe\", \"age\": 25},\n        {\"id\": 3, \"name\": \"John\", \"last_name\": \"Smith\", \"age\": 40},\n    ]\n\n\n@app.task\ndef step2(step1):\n    print(\"Compute something ....\")\n    return [item for item in step1 if item[\"age\"] \u003e 30]\n\n@app.task\ndef final(step2):\n    print(\"Return the result\")\n    return step2[0]\n\n\n\nresult = app.run()\nprint(result)\n```\n\nAfter the first run of the pipeline, the output will be cached and on second execution you will get results without\nactual run any of tasks, saving time and computational resources. However, changing code, let's say, in a `step2` \nmethod will be automatically detected by Callisto, and both steps 2 and 3 will be executed.\n\n## TO-DO\n\nThis project (and a concept in general), under heavy development. No public contract is guaranteed at this point.\n\n- [X] Add support for runtime artifacts (settings)\n- [X] Add support for multiple run with different settings\n- [ ] Terminate pipeline if task raised an exception\n- [ ] All stdout should be captured\n- [ ] Show task title and description based on the docstring\n- [ ] Let task overwrite artifacts\n- [ ] Add support for `no_cache` flag in tasks (should help when you write tasks and want to debug)\n- [ ] Come up with a better way to handle multiple variables in output\n\n\n## Development setup\n\nIn unix-like environment, you can use `make` to run pre-defined commands, like `make init` to install virtual environment, \n`make test` to run tests, and `make lint` to check code style.\n\nSee `make help` for more information.\n\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fakopdev%2Fcallisto","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fakopdev%2Fcallisto","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fakopdev%2Fcallisto/lists"}