{"id":14977150,"url":"https://github.com/alexdremov/igogo","last_synced_at":"2025-10-27T23:31:54.380Z","repository":{"id":146174845,"uuid":"617334595","full_name":"alexdremov/igogo","owner":"alexdremov","description":"Execute several jupyter cells simultaneously with beautiful output. Do not waste time waiting","archived":false,"fork":false,"pushed_at":"2023-11-10T14:57:24.000Z","size":146,"stargazers_count":5,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-08-25T21:51:36.330Z","etag":null,"topics":["ipython","ipython-magic","ipython-notebook","ipython-notebooks","jupyter","jupyter-notebook","jupyter-notebooks","jupyterlab"],"latest_commit_sha":null,"homepage":"https://pypi.org/project/igogo/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/alexdremov.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-03-22T07:08:53.000Z","updated_at":"2023-06-02T13:50:58.000Z","dependencies_parsed_at":"2024-09-11T12:32:03.394Z","dependency_job_id":"250c21e6-9f5d-49bd-9fe3-5b13fdf33b75","html_url":"https://github.com/alexdremov/igogo","commit_stats":null,"previous_names":["alexroar/igogo"],"tags_count":10,"template":false,"template_full_name":null,"purl":"pkg:github/alexdremov/igogo","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alexdremov%2Figogo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alexdremov%2Figogo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alexdremov%2Figogo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alexdremov%2Figogo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/alexdremov","download_url":"https://codeload.github.com/alexdremov/igogo/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alexdremov%2Figogo/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":281361406,"owners_count":26487881,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-27T02:00:05.855Z","response_time":61,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ipython","ipython-magic","ipython-notebook","ipython-notebooks","jupyter","jupyter-notebook","jupyter-notebooks","jupyterlab"],"created_at":"2024-09-24T13:55:11.900Z","updated_at":"2025-10-27T23:31:54.065Z","avatar_url":"https://github.com/alexdremov.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# igogo 🐎🏎️\n\nExecute several jupyter cells at the same time\n\n\u003e Have you ever just sited and watched a long-running jupyter cell?\n\u003e **Now, you can continue to work in the same notebook freely**\n\nhttps://user-images.githubusercontent.com/25539425/227176976-2bdda463-ecc9-4431-afec-6d31fbd4c214.mov\n\n---\n\n## Use Cases\n1) **You have a long-running cell, and you need to check something.\n   You can just start the second cell without interrupting a long-running cell**.\n   \u003e **Example:** you run a machine learning train loop and want to immediately save the model's weights or check metrics.\n   \u003e With `igogo` you can do so without interrupting the training.\n2) **If you need to compare the score of some function with different parameters, you can run several\n   functions at the same time and monitor results**. \n   \u003e **Example:** you have several sets of hyperparameters and want to compare them.\n   \u003e You can start training two models, monitoring two loss graphs at the same time. \n3) **Process data in chunks**. Check processed data for validity\n   \u003e **Example:** you do data processing in steps. With `igogo` you can execute several steps at the same time\n   \u003e and process data from the first processing step in the second processing step in chunks.\n   \u003e Also, you can quickly check that the first step produces the correct results\n\n## Install\n\nIgogo is available through PyPi:\n\n```bash\npip install igogo\n```\n\n## Wait, isn't it just a background job? No.\n\n- **No multithreading, no data races, no locks**.\nYou can freely operate with your notebook variables without the risk of corrupting them.\n- **Beautiful output**. When several cells execute in parallel,\nall printed data is displayed in the corresponding cell's output. No more twisted and messed out concurrent outputs.\n- **Easily cancel jobs, wait for completion, and start the new ones**.\n- **Control execution of jobs through widgets**.\n\n## Usage\n\nAt the core of igogo is collaborative execution. Jobs need to explicitly allow other jobs to execute through `igogo.yielder()`. Mind that regular cells also represent a job.\n\nPlacing `igogo.yielder()` in code that is not executed in igogo job is not a mistake. It will return immediately. So, you don't need to care about keeping `igogo.yielder()` only in igogo jobs. You can place it anywhere\n\nTo start an igogo job, you can use `%%igogo` cell magic or function decorator. \n\n```python\nimport igogo\n\n@igogo.job\ndef hello_world(name):\n    for i in range(3):\n        print(\"Hello, world from\", name)\n        \n        # allows other jobs to run while asleep\n        # also can be `igogo.yielder()`\n        igogo.sleep(1)  \n    return name\n```\n\nCall function as usual to start a job:\n\n```python\nhello_world('igogo'), hello_world('other igogo');\n```\n\nhttps://user-images.githubusercontent.com/25539425/227186815-6870e348-46e6-4086-a89b-be416c0cc1a7.mov\n\n### Configure Jobs\n\nDecorator `@igogo.job` has several useful parameters. \n\n- `kind`\\\n   Allows to set how to render output. Possible options: `text`, `markdown`, `html` Default: `text`\n- `displays`\\\n   As igogo job modify already executed cell, it needs to have spare placeholders for rich output.\n   This parameter specifies how many spare displays to spawn. Default: `1`\n- `name`\\\n   User-friendly name of igogo job.\n- `warn_rewrite`\\\n   Should warn rewriting older displays? Default: `True`\n- `auto_display_figures`\\\n   Should display pyplot figures created inside igogo automatically? Default: `True`\n\nMarkdown example:\n\nhttps://user-images.githubusercontent.com/25539425/227203729-af94582c-8fe2-40fe-a6f0-6489a374a88f.mov\n\n### Display Additional Data\n\nPyplot figures will be automatically displayed in igogo cell.\n\nYou can also use `igogo.display` inside a job to display any other content or several figures. Mind that displays must be pre-allocated by specifying displays number in `igogo.job(displays=...)`\n\n```python\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport igogo\n\ndef experiment(name, f, i):\n     x = np.linspace(0, i / 10, 100)\n     fig = plt.figure()\n     plt.plot(\n         x,\n         f(x)\n     )\n     plt.gca().set_title(name)\n     igogo.display(fig)\n     \n     fig = plt.figure()\n     plt.scatter(\n         x,\n         f(x)\n     )\n     plt.gca().set_title(name)\n     igogo.display(fig)\n     igogo.sleep(0.05)\n```\n\nAs noted in \"Configure jobs\" section, `igogo` jobs have limited number of displays.\nIf you try to display more objects than job has, warning will be shown and the oldest displays will be overwritten.\n\n### Cell Magic\n\nThe same way with `%%igogo`:\n\n```python\n%load_ext igogo\n```\n\n```python\n%%igogo\nname = 'igogo'\nfor i in range(3):\n     print(\"Hello, world from\", name)\n     igogo.sleep(1)\n```\n\n### Widgets\n\nAll executed `igogo` jobs spawn a widget that allows to kill them. Jobs are not affected by `KeyboardInterrupt`\n\n### Killing Jobs\n\nApart from killing through widgets, `igogo` jobs can be killed programmatically.\n\n- `igogo.stop()` \\\n   Can be called inside `igogo` job to kill itself.\n- `igogo.stop_all()`\\\n   Stops all running `igogo` jobs\n- `igogo.stop_latest()`\\\n   Stops the latest `igogo` job. Can be executed several times.\n- `igogo.stop_by_cell_id(cell_id)`\\\n   Kills all jobs that were launched in cell with `cell_id` (aka [5], cell_id=5).\n\nAlso, you can stop jobs of one specific function.\n\n- `hello_world.stop_all()`\\\n   Stops all `igogo` jobs created by `hello_world()`\n\n## Supported Clients\n\nCurrently, `igogo` runs fully correct on:\n\n- Jupyter Lab\n- Jupyter\n\nRuns but has problems with output from igogo jobs. Jobs are executed, but there could be problems with widgets and output:\n- VSCode. For some reason it does not update display data. Therefore, no output is produced.\n- DataSpell. It displays `[object Object]` and not output.\n- Colab. It does not support updating content of executed cells\n\n## More Examples\n\n[**Check out pretty notebooks**](https://github.com/alexdremov/igogo/tree/main/examples)\n\n---\n\n### Train model and check metrics \n\nhttps://user-images.githubusercontent.com/25539425/227651626-cba8a317-a986-4971-9639-84cdb388e2d3.mov\n\nAlso, you can modify training parameters, freeze/unfreeze layers, switch datasets, etc. All you need is to place `igogo.yielder()` in train loop.\n\n### Process data and montitor execution\n\n```python\nimport igogo\nimport numpy as np\nfrom tqdm.auto import tqdm\n%load_ext igogo\n\nraw_data = np.random.randn(100000, 100)\nresult = []\n```\n\n```python\ndef row_processor(row):\n    return np.mean(row)\n```\n\n```python\n%%igogo\nfor i in tqdm(range(len(raw_data))):\n    result.append(row_processor(raw_data[i]))\n    igogo.yielder()\n```\n\n```python\nresult[-1]\n```\n\n### Process data in chunks\n\n```python\nimport igogo\nimport numpy as np\nfrom tqdm.auto import tqdm\n%load_ext igogo\n\nraw_data = np.random.randn(5000000, 100)\n\nigogo_yield_freq = 32\nigogo_first_step_cache = []\n\nresult = []\n```\n\n```python\n%%igogo\n\nfor i in tqdm(range(len(raw_data))):\n    processed = np.log(raw_data[i] * raw_data[i])\n    igogo_first_step_cache.append(processed)\n    \n    if i \u003e 0 and i % igogo_yield_freq == 0:\n        igogo.yielder()  # allow other jobs to execute\n```\n\n```python\n%%igogo\n\nfor i in tqdm(range(len(raw_data))):\n    while i \u003e= len(igogo_first_step_cache):  # wait for producer to process data\n        igogo.yielder()\n    \n    result.append(np.mean(igogo_first_step_cache[i]))\n    \n```\n\nhttps://user-images.githubusercontent.com/25539425/227224077-a3ce664c-cb52-4aa2-a3fe-71ac5a03cdeb.mov\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falexdremov%2Figogo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Falexdremov%2Figogo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falexdremov%2Figogo/lists"}