{"id":25535673,"url":"https://github.com/jku-vds-lab/loops","last_synced_at":"2026-01-29T02:30:17.142Z","repository":{"id":64950796,"uuid":"572877732","full_name":"jku-vds-lab/loops","owner":"jku-vds-lab","description":"Loops is a JupyterLab extension to support iterative and exploratory data analysis in computational notebooks.","archived":false,"fork":false,"pushed_at":"2024-04-09T11:41:49.000Z","size":8391,"stargazers_count":1,"open_issues_count":20,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2024-04-14T17:24:56.029Z","etag":null,"topics":["data-analysis","data-science","data-visualization","jupyter","jupyter-notebook","notebook","provenance"],"latest_commit_sha":null,"homepage":"https://jku-vds-lab.at/publications/2024_loops/","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jku-vds-lab.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2022-12-01T08:16:38.000Z","updated_at":"2024-04-18T07:34:57.781Z","dependencies_parsed_at":"2024-03-04T20:04:45.293Z","dependency_job_id":"9b75b77f-140d-439f-9aa3-a341cfd5b85b","html_url":"https://github.com/jku-vds-lab/loops","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jku-vds-lab%2Floops","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jku-vds-lab%2Floops/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jku-vds-lab%2Floops/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jku-vds-lab%2Floops/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jku-vds-lab","download_url":"https://codeload.github.com/jku-vds-lab/loops/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239533071,"owners_count":19654616,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-analysis","data-science","data-visualization","jupyter","jupyter-notebook","notebook","provenance"],"created_at":"2025-02-20T04:22:56.719Z","updated_at":"2026-01-29T02:30:17.086Z","avatar_url":"https://github.com/jku-vds-lab.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# loops\n\n[![Github Actions Status](https://github.com/jku-vds-lab/loops/workflows/Build/badge.svg)](https://github.com/jku-vds-lab/loops/actions/workflows/build.yml)[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/jku-vds-lab/loops/main?labpath=notebooks/)\n\nLoops is a JupyterLab extension to support iterative and exploratory data analysis in computational notebooks.\n\nLoops automatically tracks the notebook's history and visualizes it next to the notebook.\nLoops shows the evolution of the notebook over time and highlights differences between versions to reveal the impact of changes made within a notebook.\nLoops visualizes differences in code, markdown, tables, visualizations, and images.\nFor a quick overview of loops, see our preview video on YouTube:\n\n[\u003cimg src=\"https://img.youtube.com/vi/jCUwLm5wfNo/maxresdefault.jpg\" width=50% height=50%\u003e](https://www.youtube.com/watch?v=jCUwLm5wfNo)\n\nTry loops yourself on Binder with two example notebooks for which the analysis process has been recorded. Loops is part of JuypterLab's sidebar and can be opened from there.\n\n- Use Case 1: Concert Data Analaysis [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/jku-vds-lab/loops/main?labpath=notebooks/Use%20Case%201.ipynb)\n- Use Case 2: What-If Analysis on Cancer Patient Data [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/jku-vds-lab/loops/main?labpath=notebooks/Use%20Case%202.ipynb)\n\nThe Use Case data and notebooks are also available on [OSF](https://osf.io/hxuak/) to try loops in a local environment (see [Usage](#usage)).\n\nTo learn more about loops, read our [paper](https://jku-vds-lab.at/publications/2024_loops/).  \nAbstract:\n\n\u003e Exploratory data science is an iterative process of obtaining, cleaning, profiling, analyzing, and interpreting data. This cyclical way of working creates challenges within the linear structure of computational notebooks, leading to issues with code quality, recall, and reproducibility. To remedy this, we present Loops, a set of visual support techniques for iterative and exploratory data analysis in computational notebooks. Loops leverages provenance information to visualize the impact of changes made within a notebook. In visualizations of the notebook provenance, we trace the evolution of the notebook over time and highlight differences between versions. Loops visualizes the provenance of code, markdown, tables, visualizations, and images and their respective differences. Analysts can explore these differences in detail in a separate view. Loops not only improves the reproducibility of notebooks but also supports analysts in their data science work by showing the effects of changes and facilitating comparison of multiple versions. We demonstrate our approach's utility and potential impact in two use cases and feedback from notebook users from various backgrounds.\n\n## Usage\n\nRequires JupyterLab \u003e= 4.0.0.\n\nYou can install loops with JupyterLab's extension manager:\n![image](https://github.com/jku-vds-lab/loops/assets/10337788/ec26c434-c4b2-4610-b5be-6720014a42d4)\n\nOr from the command line using pip:\n\n```bash\npip install loops-diff\n```\n\n## Feedback\n\nYour comments and feedback are welcome. Write an email to klaus.eckelt@jku.at and let us know what you think!  \nIf you have discovered an issue or have a feature suggestion, feel free to [create an issue on GitHub](https://github.com/jku-vds-lab/loops/issues).\n\n## Citing Loops\n\nKlaus Eckelt, Kiran Gadhave, Alexander Lex, Marc Streit.  \n**Loops: Leveraging Provenance and Visualization to Support Exploratory Data Analysis in Notebooks**.  \nIEEE Transactions on Visualization and Computer Graphics (IEEE VIS '24, to appear), doi:10.31219/osf.io/79eyn, 2024.\n\n```\n@article{2024_loops,\n    title = {Loops: Leveraging Provenance and Visualization to Support Exploratory Data Analysis in Notebooks},\n    author = {Klaus Eckelt and Kiran Gadhave and Alexander Lex and Marc Streit},\n    journal = {IEEE Transactions on Visualization and Computer Graphics (IEEE VIS '24, to appear)},\n    doi = {10.31219/osf.io/79eyn},\n    url = {https://doi.org/10.31219/osf.io/79eyn},\n    year = {2024}\n}\n```\n\n---\n\n## Contributing\n\nThere are two ways to set up _loops_ for development:\n\n- with [VS Code devContainers](https://code.visualstudio.com/docs/devcontainers/containers) (recommended), which sets up Jupyter and the dependencies in a container.\n- with a local setup, the default way for JupyterLab extension. Works with every code editor.\n\n### Development in DevContainer\n\n#### Requirements\n\n- [VS Code](https://code.visualstudio.com/)\n- [Dev Containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers) for VS Code\n- [Docker](https://www.docker.com/products/docker-desktop/)\n\nSee the [official Dev Containers Tutorial](https://code.visualstudio.com/docs/devcontainers/tutorial) for more detailed instructions and alternatives.\n\nA [devcontainer.json file](https://code.visualstudio.com/docs/devcontainers/containers) is provided that uses the official [scipy-notebook container](https://jupyter-docker-stacks.readthedocs.io/en/latest/using/selecting.html#jupyter-scipy-notebook) for development (includes JupyterLab, numpy, pandas, matplotlib, seaborn, altair, scikit-learn, and more).\n\n#### Setup\n\nWith the above requirements fullfilled, you will see the following popup when opening this project in VS Code:\n![VS Code popup](https://user-images.githubusercontent.com/10337788/207567396-660f5e3e-3e0c-4cd6-8fcb-e4cf679860cc.png)\n\nAlternatively, you can also reopen the project in a devcontainer via the command prompt:\n![VS Code command prmpt](https://github.com/jku-vds-lab/loops/assets/10337788/e2f624a0-9238-4d32-856b-7e47c937a496)\n\nBy reopening in the container, you will get an environment with Jupyter Lab and the packages from the docker image and it will also install all dependencies of the extension as well as the extension itself. Therefore, this process will take a while when doing it for the first time. You can watch the set up process by opening the log in the terminal. When the extension is installed, the terminal should look similar to this:\n\n![image](https://github.com/jku-vds-lab/loops/assets/10337788/16f8eb34-6f0d-45d1-aa5b-17772feab31a)\n\nAll you need to do, is running `jlpm watch` in the VS Code terminal afterwards so that the extension gets updated when you make code changes.  \nThe terminal can also be used to add further python packages.\n\n### Local Development\n\nNote: You will need NodeJS to build the extension package.\n\n\u003e Tested with Node 18 and Python 3.9\n\nYou also may want to create a virtual environment, i.e.\n\n```bash\npython -m venv env\nsource env/bin/activate\n\n# Install Jupyterlab and any other python packages you want to use\npython -m pip install \"jupyterlab\u003e=4.0.0\"\n```\n\nThe `jlpm` command is JupyterLab's pinned version of\n[yarn](https://yarnpkg.com/) that is installed with JupyterLab. You may use\n`yarn` or `npm` in lieu of `jlpm` below.\n\n```bash\n# Clone the repo to your local environment\n# Change directory to the loops directory\n# Install package in development mode\npython -m pip install -e \".\"\n# Link your development version of the extension with JupyterLab\npython -m jupyter labextension develop . --overwrite\n# Rebuild extension Typescript source after making changes\njlpm build\n```\n\nYou can watch the source directory and run JupyterLab at the same time in different terminals to watch for changes in the extension's source and automatically rebuild the extension.\n\n```bash\n# Watch the source directory in one terminal, automatically rebuilding when needed\njlpm watch\n# Run JupyterLab in another terminal\npython -m jupyter lab\n```\n\nWith the watch command running, every saved change will immediately be built locally and available in your running JupyterLab. Refresh JupyterLab to load the change in your browser (you may need to wait several seconds for the extension to be rebuilt).\n\nBy default, the `jlpm build` command generates the source maps for this extension to make it easier to debug using the browser dev tools. To also generate source maps for the JupyterLab core extensions, you can run the following command:\n\n```bash\npython -m jupyter lab build --minimize=False\n```\n\n### Testing the extension\n\n#### Frontend tests\n\nThis extension is using [Jest](https://jestjs.io/) for JavaScript code testing.\n\nTo execute them, execute:\n\n```sh\njlpm\njlpm test\n```\n\n#### Integration tests\n\nThis extension uses [Playwright](https://playwright.dev/) for the integration tests (aka user level tests).\nMore precisely, the JupyterLab helper [Galata](https://github.com/jupyterlab/jupyterlab/tree/master/galata) is used to handle testing the extension in JupyterLab.\n\nMore information are provided within the [ui-tests](./ui-tests/README.md) README.\n\n### Packaging the extension\n\nSee [RELEASE](RELEASE.md)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjku-vds-lab%2Floops","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjku-vds-lab%2Floops","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjku-vds-lab%2Floops/lists"}