{"id":40584004,"url":"https://github.com/analysiscenter/nbtools","last_synced_at":"2026-01-21T02:38:54.644Z","repository":{"id":41814179,"uuid":"449323788","full_name":"analysiscenter/nbtools","owner":"analysiscenter","description":"Utilities for monitoring and interacting with Jupyter Notebooks","archived":false,"fork":false,"pushed_at":"2025-10-21T11:18:45.000Z","size":9395,"stargazers_count":38,"open_issues_count":4,"forks_count":1,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-10-21T12:11:05.942Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://analysiscenter.github.io/nbtools/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/analysiscenter.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2022-01-18T14:46:49.000Z","updated_at":"2025-07-04T06:14:08.000Z","dependencies_parsed_at":"2022-08-11T18:11:40.020Z","dependency_job_id":"0b1e0ca9-24c7-4d39-b6fd-e40eaf191144","html_url":"https://github.com/analysiscenter/nbtools","commit_stats":{"total_commits":53,"total_committers":3,"mean_commits":"17.666666666666668","dds":"0.26415094339622647","last_synced_commit":"4a9199b7a73cb25963495e21b868a6c38a3bc2d9"},"previous_names":[],"tags_count":10,"template":false,"template_full_name":null,"purl":"pkg:github/analysiscenter/nbtools","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/analysiscenter%2Fnbtools","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/analysiscenter%2Fnbtools/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/analysiscenter%2Fnbtools/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/analysiscenter%2Fnbtools/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/analysiscenter","download_url":"https://codeload.github.com/analysiscenter/nbtools/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/analysiscenter%2Fnbtools/sbom","scorecard":{"id":191066,"data":{"date":"2025-08-11","repo":{"name":"github.com/analysiscenter/nbtools","commit":"03421f66d63cb883812f24ea6c9d62dafc64c846"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":2.8,"checks":[{"name":"Maintained","score":0,"reason":"1 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Code-Review","score":4,"reason":"Found 4/10 approved changesets -- score normalized to 4","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Dangerous-Workflow","score":0,"reason":"dangerous workflow patterns detected","details":["Warn: script injection with untrusted input ' github.head_ref ': .github/workflows/test-install.yml:61"],"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Warn: topLevel 'contents' permission set to 'write': .github/workflows/doc.yml:6","Warn: no topLevel permission defined: .github/workflows/release.yml:1","Warn: no topLevel permission defined: .github/workflows/status.yml:1","Warn: no topLevel permission defined: .github/workflows/test-install.yml:1","Info: no jobLevel write permissions found"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/doc.yml:12: update your workflow using https://app.stepsecurity.io/secureworkflow/analysiscenter/nbtools/doc.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/doc.yml:13: update your workflow using https://app.stepsecurity.io/secureworkflow/analysiscenter/nbtools/doc.yml/master?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/doc.yml:22: update your workflow using https://app.stepsecurity.io/secureworkflow/analysiscenter/nbtools/doc.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/release.yml:13: update your workflow using https://app.stepsecurity.io/secureworkflow/analysiscenter/nbtools/release.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/status.yml:12: update your workflow using https://app.stepsecurity.io/secureworkflow/analysiscenter/nbtools/status.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/test-install.yml:24: update your workflow using https://app.stepsecurity.io/secureworkflow/analysiscenter/nbtools/test-install.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/test-install.yml:27: update your workflow using https://app.stepsecurity.io/secureworkflow/analysiscenter/nbtools/test-install.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/test-install.yml:56: update your workflow using https://app.stepsecurity.io/secureworkflow/analysiscenter/nbtools/test-install.yml/master?enable=pin","Warn: pipCommand not pinned by hash: .github/workflows/doc.yml:16","Warn: pipCommand not pinned by hash: .github/workflows/doc.yml:17","Warn: pipCommand not pinned by hash: .github/workflows/release.yml:17","Warn: pipCommand not pinned by hash: .github/workflows/status.yml:16","Warn: pipCommand not pinned by hash: .github/workflows/test-install.yml:33","Warn: pipCommand not pinned by hash: .github/workflows/test-install.yml:34","Warn: pipCommand not pinned by hash: .github/workflows/test-install.yml:62","Warn: pipCommand not pinned by hash: .github/workflows/test-install.yml:63","Warn: pipCommand not pinned by hash: .github/workflows/test-install.yml:64","Info:   0 out of   7 GitHub-owned GitHubAction dependencies pinned","Info:   0 out of   1 third-party GitHubAction dependencies pinned","Info:   0 out of   9 pipCommand dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: Apache License 2.0: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":-1,"reason":"internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration","details":null,"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 27 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-16T20:41:02.690Z","repository_id":41814179,"created_at":"2025-08-16T20:41:02.690Z","updated_at":"2025-08-16T20:41:02.690Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28623737,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-20T23:49:58.628Z","status":"online","status_checked_at":"2026-01-21T02:00:08.227Z","response_time":86,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-01-21T02:38:54.010Z","updated_at":"2026-01-21T02:38:54.637Z","avatar_url":"https://github.com/analysiscenter.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# NBTools\nCollection of tools for monitoring running Jupyter Notebooks and interacting with them.\n\nThe installation should be as easy as:\n```\npip install py-nbtools\n```\n\n\n## **NBstat**\n\nThe main tool of this package is **nbstat** / **nbwatch** command line utility. It is added at installation and shows the detailed resource utilization for each process of each running Jupyter Notebook. A gif is worth a thousand words:\n\n\u003cimg src=\"images/nbwatch.gif\" width=\"90%\"/\u003e\n\nFor more information, check out the full [user documentation:](nbtools/nbstat/README.md) explanation of different table views, command line options and ready-to-use snippets.\n\n\n### Troubleshooting: PID namespaces, user permissions and zombie processes\nA [known problem](https://github.com/NVIDIA/nvidia-docker/issues/179) of NVIDIA drivers is that **nvidia-smi** reports PIDs of processes on devices in the global namespace, not in the container namespace, which does not allow to match PIDs of container processes to their device PIDs. There are a few workarounds:\n* [recommended] pass `--pid=host` flag to `docker run`.\n* patch NVIDIA driver to handle PID namespaces correctly.\n* [Linux only] fallback on manually inspecting */proc/PID/* files to identify the host PID for each process inside of the container.\n\nWhile `nbstat` provides several fallbacks for `Linux` containers (and intend to provide support for more environments over time), the bullet-proof way is to use `--pid=host` option for `docker run`. Adding it resolves most of the issues immediately.\n\nOne more thing that sometimes happens to NVIDIA devices is zombie processes: by incorrectly terminating a GPU-using process you can end up in a situation where device memory is held by not-existing process. As far as I know, there are no ways of killing them without rebooting, and `nbstat` just marks such processes with red color.\n\nIn order to inspect certain properties of processes, we rely on having all necessary permissions already provided at command run. `nbstat` has some fallbacks for some attributes, and I currently work on improving error handling in cases of denied access to files.\n\n\n\n### Contribute\nIf you are interested to contribute, check out the [developer/contributor page.](nbtools/nbstat/DEV.md) It contains detailed description about inner workings of the library, my design choices and motivation behind them, as well as discussion of complexities along the way.\n\n## Library\nOther than `nbstat / nbwatch` monitoring utilities, this library provides a few useful tools for working with notebooks and GPUs.\n\n\n### **pylint_notebook**\nShamelessly taken from [pylint page:](https://pylint.pycqa.org/en/latest/)\n\nFunction that checks for errors in Jupyter Notebooks with Python code, tries to enforce a coding standard and looks for code smells. It can also look for certain type errors, it can recommend suggestions about how particular blocks can be refactored and can offer you details about the code's complexity.\n\nUsing it as easy as:\n```python\nfrom nbtools import pylint_notebook\npylint_notebook(path_to_ipynb,             # If not provided, use path to the current notebook\n                disable='invalid-name',    # Disable specified Pylint checks. Can be a list.\n                enable='import-error')     # Enable  specified Pylint checks. Can be a list.\n```\n\nUnder the hood, it converts `.ipynb` notebook to `.py` script, creates a custom `.pylintrc` configuration, runs the `pylint` and removes all temporary files. Learn more about its usage in the [tutorial.](tutorials/NBstat.ipynb)\n\n### **exec_notebook**\nProvides a `eval`-like interface for running Jupyter Notebooks programmatically. We use it for running interactive tests, that are easy to work with: in case of any failures, one can jump right into fixing it with an already set-up environment.\n\n```python\nfrom nbtools import exec_notebook\nexec_notebook(path_to_ipynb,                       # Which notebook to run\n              out_path_ipynb,                      # Where to save result\n              inputs={'learning_rate': 0.05,},     # Pass variables to notebook\n              outputs=['accuracy'])                # Extract variables from notebook\n```\n\n\n### **set_gpus, free_gpus**\nSelect free device(s) and set `CUDA_VISIBLE_DEVICES` environment variable so that the current process sees only them.\n\nEliminates an enormous amount of bugs and unexpected behaviors related to GPU usage.\n\n```python\nfrom nbtools import set_gpus, free_gpus\nused_gpus = set_gpus(n=2,                # Number of devices to set.\n                     min_free_memory=0.7,# Minimum amount of free memory on device to consider free.\n                     max_processes=3)    # Maximum amount of  processes  on device to consider free.\nfree_gpus(used_gpus)                     # Kill all processes on selected GPUs. Useful at teardown.\n```\n\n### Other functions\n```python\nfrom nbtools import (in_notebook,         # Return True if executed inside of Jupyter Notebook\n                     get_notebook_path,   # If executed in Jupyter Notebook, return its absolute path\n                     get_notebook_name,   # If executed in Jupyter Notebook, return its name\n                     notebook_to_script)  # Convert Jupyter Notebook to an executable Python script.\n                                          # Works well with magics and command line executions.\n```\n\n\n## Goals\nThis library started as a container of tools, that I came across / developed in my years as an ML researcher. As some of the functions survived multiple refactoring iterations, I decided to share the library so it is easier to perfect them and test in different environments.\n\nAnother goal of the project is to show how to communicate with Jupyter API on real world examples: instead of reading through a number of stackoverflow threads, you can find the same information collected in one place and get a rough understanding of what is possible with it and what is not.\n\n## Acknowledgements\nThe **nbstat** module builds on [**gpustat**](https://github.com/wookayin/gpustat) package. Using the **gpustat** for years gave me an idea about possible improvements, which are implemented in this library. While the implementation is different, reading through the code of **gpustat** was essential for development.\n\nAnimated GIFs are created by using [Terminalizer](https://github.com/faressoft/terminalizer): aside from the usual problems with installation, the tool itself is amazing.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fanalysiscenter%2Fnbtools","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fanalysiscenter%2Fnbtools","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fanalysiscenter%2Fnbtools/lists"}