{"id":22546509,"url":"https://github.com/deephaven/deephaven-plugins","last_synced_at":"2026-02-09T18:07:51.158Z","repository":{"id":194837263,"uuid":"688538805","full_name":"deephaven/deephaven-plugins","owner":"deephaven","description":"Deephaven Plugins","archived":false,"fork":false,"pushed_at":"2025-04-07T18:16:20.000Z","size":45280,"stargazers_count":12,"open_issues_count":184,"forks_count":16,"subscribers_count":8,"default_branch":"main","last_synced_at":"2025-04-07T19:28:41.105Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/deephaven.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-09-07T14:55:49.000Z","updated_at":"2025-04-07T18:16:25.000Z","dependencies_parsed_at":"2023-12-20T23:59:54.863Z","dependency_job_id":"f6cc9cc1-67c9-4cf0-ba03-4323e4b6001a","html_url":"https://github.com/deephaven/deephaven-plugins","commit_stats":null,"previous_names":["deephaven/deephaven-plugins"],"tags_count":61,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deephaven%2Fdeephaven-plugins","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deephaven%2Fdeephaven-plugins/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deephaven%2Fdeephaven-plugins/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deephaven%2Fdeephaven-plugins/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/deephaven","download_url":"https://codeload.github.com/deephaven/deephaven-plugins/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247719025,"owners_count":20984676,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-07T15:08:02.881Z","updated_at":"2026-02-09T18:07:51.140Z","avatar_url":"https://github.com/deephaven.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Plugin modules\n\nThe `plugins` folder contains internally developed Plugin modules.\nEach folder contains either a python plugin, a javascript plugin, or both.\nThe python plugin is housed in the top level of each plugin folder, and the javascript plugin is housed in the `src/js`\nfolder within each individual plugin folder.\n\n## Python Plugins\n\nEach python plugin should have code stored in a non-colliding path within the `deephaven` folder. Generally, this path\nwill be `deephaven/plugin/\u003cpluginName\u003e`, but it may be different depending on the plugin. For example, the `plotly-express`\nplugin is housed in `deephaven/plot/express` so that it has a structure that mirrors Plotly Express.\n\nEach python plugin should also have at least the following:\n\n- `pyproject.toml`\n- `setup.cfg`\n- `README.md`\n- Independently versioned\n- A github workflow that builds the plugin and publishes it to pypi\n- Black formatting applied to all python files\n\n## JS Plugins\n\nEach js plugin should be the following:\n\n- Based off the [JS Module Plugin template](https://github.com/deephaven/deephaven-js-plugin-template/)\n- Package name `@deephaven/js-plugin-\u003cfolderName\u003e`\n- Independent versioning, `npm install`, `npm run build`\n- Exported as a CJS bundle\n- Externalize `react`, `react-dom`, `redux`, `react-redux`, and any appropriate `@deephaven/*` packages if used\n  - Add it as a `rollupOptions.external` in `vite.config.ts`\n\n## Development\n\nStart by setting up the python venv and pre-commit hooks.\n\n### Pre-commit hooks/Python formatting\n\nFor a more automated development experience, see the [plugin_builder.py](#using-plugin_builderpy) section for a script that can help automate some of the setup steps.\n\nBlack and blacken-docs formatting, pyright type checking, and ruff linting is setup through a pre-commit hook.\nTo install the pre-commit hooks, run the following commands from the root directory of this repo:\n\n```shell\npython -m venv .venv\nsource .venv/bin/activate\npip install --upgrade -r requirements.txt\npre-commit install\n```\n\nThis will setup a venv, activate it, and install the pre-commit hooks. The hooks will run on every commit.\nYou can verify that pre-commit is setup by testing with the following:\n\n```shell\npre-commit run --all-files\n```\n\nAll steps should pass.\n\nTo bypass the pre-commit hook, you can commit with the `--no-verify` flag, for example:\n\n```shell\ngit commit --no-verify -m \"commit message\"`\n```\n\n### Running end-to-end tests\n\nWe use [Playwright](https://playwright.dev/) for end-to-end tests. We test against Chrome, Firefox, and Webkit (Safari). Snapshots from E2E tests are only run against Linux so they can be validated in CI.\n\nYou should be able to pass arguments to these commands as if you were running Playwright via CLI directly. For example, to test only `matplotlib.spec.ts` you could run `npm run e2e:docker -- ./tests/matplotlib.spec.ts`, or to test only `matplotlib.spec.ts` in Firefox, you could run `npm run e2e:docker -- --project firefox ./tests/matplotlib.spec.ts`. See [Playwright CLI](https://playwright.dev/docs/test-cli) for more details.\n\nIt is highly recommended to use `npm run e2e:docker` (instead of `npm run e2e`) as CI also uses the same environment. You can also use `npm run e2e:update-snapshots` to regenerate snapshots in said environment. If you only need to update the snapshots for a specific test, you can run `npm run e2e:update-snapshots -- ./tests/ui.spec.ts` to update only the snapshots for that test and save some time.\n\nIf you want to run tests locally (which may be easier for debugging or creating new tests), run Playwright in [UI Mode](https://playwright.dev/docs/test-ui-mode) with `npm run e2e:ui`. This will allow you to run each test individually, see the browser as it runs it, inspect the console, evaluate locators, etc. **Note**: There may be a permissions issue starting up Playwright in local mode after running it in docker. If you encounter this, delete the previous `test-results` folder using `sudo rm -rf test-results`.\n\n### Running Python tests\n\nThe [venv setup](#pre-commit-hookspython-formatting) steps will also set up `tox` to run tests for the python plugins that support it.\nNote that `tox` sets up an isolated environment for running tests.  \nBe default, `tox` will run against Python 3.9, which will need to be installed on your system before running tests.\nYou can run tests with the following command from the `plugins/\u003cplugin\u003e` directory:\n\n```shell\ntox -e py\n```\n\nYou can also run tests against a specific version of python by appending the version to `py`  \nThis assumes that the version of Python you're targeting is installed on your system.  \nFor example, to run tests against Python 3.12, run:\n\n```shell\ntox -e py3.12\n```\n\n### Running plugin against deephaven-core\n\n#### Building Python plugins for development\n\nBuild and install the plugin wheels for the plugins, plotly and matplotlib in this example, into the deephaven-core venv.\nSee READMEs in the directories of the python plugins you're working with for specific packages to install for development with that plugin.\n\nYou can build the wheels using the following commands (or similar for other plugins) from the root directory of this repo:\n\n```\npython -m build --wheel plugins/matplotlib\npython -m build --wheel plugins/plotly\n```\n\n#### Building JS plugins for development\n\nRun `npm install` to install js dependencies.\n\nYou can build the js plugin(s) in watch mode from the root directory of this repo by using the following commands:\n\n```shell\n# build all plugins in watch mode and serves the plugins directory\nnpm start\n```\n\nThis will serve the plugins using Vite's dev server. You can optionally provide a `--scope` argument to filter which .js plugins will be built in watch mode:\n\ne.g. To run all packages containing \"theme\" in the name:\n\n```shell\n# include a scope to filter which plugins to build in watch mode\nnpm start -- --scope *theme*\n```\n\nAlternatively, you can also run individual plugins in watch mode. Note that this will only build the plugin and won't start the dev server.\n\n```shell\n# build a single plugin in watch mode\ncd plugins/plugin\nnpm start\n```\n\nNote that if you are mapping the `plugins` folder directly via DHC start options, the plugins dev server won't actually be used, and you will need to restart the deephaven-core server each time a change is made for the change to be picked up.\n\n##### Serve Plugins\n\nRunning `npm start` will also will also serve the `plugins` directory using Vite's local dev server. The default host + port is `http://localhost:4100`, but the port can be configured via the `PORT` env variable.\n\nDHC and DHE can be configured when running locally to target the local `plugins` server. This has the benefit of not requiring a server restart when developing plugins. See [DHC](https://github.com/deephaven/web-client-ui/blob/main/README.md#local-plugin-development) or [DHE](https://github.com/deephaven-ent/iris/blob/rc/grizzly/web/client-ui/README.md#local-plugin-development) README for details on using this configuration.\n\n#### Running deephaven-core\n\n##### Build deephaven-core\n\nBuild [deephaven-core](https://github.com/deephaven/deephaven-core) using the directions [here](https://deephaven.io/core/docs/getting-started/launch-build/#build-and-run-deephaven).\n\n##### Install python plugin wheels\n\nThen, install the python plugin wheels for the plugins, plotly and matplotlib in this example, into the deephaven-core venv.\nSee READMEs in the directories of the python plugins you're working with for specific packages to install for development with that plugin.\n\nYou can build the wheels using the following commands (or similar for other plugins) from the root directory of this repo:\n\n```\n\npython -m build --wheel plugins/matplotlib\npython -m build --wheel plugins/plotly\n\n```\n\nSubstitute in your local wheel locations for the wheels in the following command.\nNote that `\u003cdeephaven-plugins-path\u003e` is the path to this repo.\n\n```\n\npip install \u003cdeephaven-plugins-path\u003e/plotly/plugins/dist/deephaven_plugin_plotly-0.0.1.dev2-py3-none-any.whl \u003cdeephaven-plugins-path\u003e/plugins/matplotlib/dist/deephaven_plugin_matplotlib-0.1.1-py3-none-any.whl\n\n```\n\nIf installing multiple wheels, you can use the following shorthand to install all built wheels:\n\n```\n\npip install \u003cdeephaven-plugins-path\u003e/plugins/_/dist/_.whl\n\n```\n\nIf you're reinstalling the python wheels without a version bump (generally for the purpose of development), you'll want to add the `--force-reinstall` tag. The `--no-deps` tag is also recommended as `--force-reinstall` will update all the dependencies as well, which is generally unnecessary.\nFor example, on reinstalls the above command becomes\n\n```\n\npip install --force-reinstall --no-deps \u003cdeephaven-plugins-path\u003e/plugins/_/dist/_.whl\n\n```\n\n##### Start deephaven-core\n\nFinally, start up Deephaven with the appropriate js-plugin flags using the path to your deephaven-plugins repo. For example, to start with the matplotlib and plotly plugins, start the server with the following command:\n\n```\n\nSTART_OPTS=\"-Ddeephaven.jsPlugins.@deephaven/js-plugin-matplotlib=\u003cdeephaven-plugins-path\u003e/plugins/matploltib/src/js -Ddeephaven.jsPlugins.@deephaven/js-plugin-plotly=\u003cdeephaven-plugins-path\u003e/plugins/plotly/src/js\" ./gradlew server-jetty-app:run\n\n```\n\nThe Deephaven IDE can then be opened at http://localhost:10000/ide/, with your plugins ready to use.\n\n### Running with Docker container\n\nInstead of running deephaven-core from source and building all plugins yourself, you can run a docker container that automatically builds the plugins and installs them in an instance of deephaven-core, then serving it up at http://localhost:10000. JS Plugins are specified in [./docker/config/deephaven.prop](./docker/config/deephaven.prop) as to which ones are loaded. Run `npm run docker` to start up the docker container, or just run `docker compose up --build` if you do not have `npm` installed. It will open at port 10000 by default, and use the demo data from [./docker/data](./docker/data) as the data folder.\nIf you wish to change the port it opens on, you can specify the `DEEPHAVEN_PORT` environment variable. For example, to open on port 11000, you would run `DEEPHAVEN_PORT=11000 npm run docker`.\nIf you wish to customize what data is used for the docker container, you can create a [docker-compose.override.yml file](https://docs.docker.com/compose/multiple-compose-files/merge/) to override the default values. For example, if you want to use `/path/to/mydata/` as the data folder instead of the default, you would add a `volumes` property to your docker-compose.override.yml:\n\n```yml\nversion: '3'\n\nservices:\n  deephaven-plugins:\n    volumes:\n      # Specifying a data volume here will override the default data folder, and you will not be able to access the default data files (such as the demo data)\n      - /path/to/mydata/:/data\n```\n\n### Using plugin_builder.py\n\nThe `tools/plugin_builder.py` script is a utility script that makes common plugin development cases easier.\nThe tool uses `click` for command line argument parsing and `watchdog` for file watching.  \nSkip the venv setup if you already have one\n\n```shell\npython -m venv .venv\nsource .venv/bin/activate\npip install --upgrade -r requirements.txt\npip install click watchdog\n```\n\nThe script can then be used to help set up your venv.\nThis command will setup the basic dependencies for building plugins:\n\n```shell\npython tools/plugin_builder.py --configure=min\n```\n\nThis command will setup the basic dependencies, plus optional ones for building docs and running the server:\n\n```shell\npython tools/plugin_builder.py --configure=full\n```\n\nThe simplest way to use the script is to run it with no arguments. This will build and install all plugins:\n\n```shell\npython tools/plugin_builder.py\n```\n\nTo target a specific plugin or plugins, pass the name or names of the plugins as arguments:\n\n```shell\npython tools/plugin_builder.py plotly-express ui\n```\n\nThis targeting works for all commands that target the plugins directly, such as `--docs` or `--install`.\n\nTo build docs, pass the `--docs` flag.  \nFirst install the necessary dependencies (if setup with `--configure=full` this is already done)\n\n```shell\npip install -r sphinx_ext/sphinx-requirements.txt\n```\n\nThis example builds the docs for the `ui` plugin:\n\n```shell\npython tools/plugin_builder.py --docs ui\n```\n\nIt is necessary to install the latest version of the plugin you're building docs for before building the docs themselves.  \nRun with `--install` or `--reinstall` to install the plugin (depending on if you're installing a new version or not)\nbefore building the docs.\n\n```shell\npython tools/plugin_builder.py --docs --install ui\n```\n\nAfter the first time install, you can drop the `--install` flag and just run the script with `--docs` unless you have plugin changes.\n\nYou can also re-generate the snapshots for the docs by passing the `--snapshots` flag. This should be done when new code blocks are added to the docs.\n\nThis example will build the docs for the `ui` plugin and re-generate the snapshots:\n\n```shell\npython tools/plugin_builder.py --docs --snapshots ui\n```\n\nTo run the server, pass the `--server` flag.  \nFirst install `deephaven-server` if it is not already installed (if setup with `--configure=full` this is already done):\n\n```shell\npip install deephaven-server\n```\n\nThis example reinstalls the `plotly-express` plugin, then starts the server:\n\n```shell\npython tools/plugin_builder.py --reinstall --server plotly-express\n```\n\nReinstall will force reinstall the plugins (but only the plugins, not the dependencies), which is useful if there are changes to the plugins but without a bumped version number.\n\nTo run the server with specific args, pass the `--server-arg` flag.  \nBy default, the server is passed the `--no-browser` flag, which will prevent the server from opening a browser window.  \nThis example will override that default and open the browser:\n\n```shell\npython tools/plugin_builder.py --server-arg --browser\n```\n\nSimilar to other arguments, this argument can be shortened to `-sa`.  \nThis example changes the port and psk and reinstalls the `ui` plugin before starting the server:\n\n```shell\npython tools/plugin_builder.py -r -sa --port=9999 -sa --jvm-args=\"-Dauthentication.psk=mypsk\" ui\n```\n\nThe js plugins can be built with the `--js` flag. This will build all js plugins or target specific ones if specified.\nThis example reinstalls the `ui` plugin with js, and starts the server with shorthand flags.\n\n```shell\npython tools/plugin_builder.py --js -r -s ui\n```\n\nEnable `watch` mode with the `--watch` flag. This will watch the project for changes and rerun the script with the same arguments.  \nNote that when using `--watch`, the script will not exit until stopped manually.\nFor example, to watch the `plotly-express` plugin for changes and rebuild the docs when changes are made:\n\n```shell\npython tools/plugin_builder.py --docs --watch plotly-express\n```\n\nThis example reinstalls the `ui` plugin with js, starts the server, and watches for changes.\n\n```shell\npython tools/plugin_builder.py -jrsw ui\n```\n\n### Previewing Docs\n\nTo preview the docs, run the following command from the root directory of this repo:\n\n```shell\nnpm run docs\n```\n\nThis will use the source directories and serve the preview docs at `http://localhost:3001`. Note that this will not include the API reference docs which are only part of the fully built docs with `plugin_builder.py`.\n\nTo build and preview docs from the `build` directory, run the following commands:\n\n```shell\npython tools/plugin_builder.py -d ui plotly-express\nBUILT=true npm run docs\n```\n\n### Snapshotting docs with npm\n\nYou can also use `npm` to snapshot the docs instead of using the `--snapshots` flag in `plugin_builder.py`. Just run `npm run update-doc-snapshots` from the root directory of this repo. This will run the snapshotting script and update the snapshots in the `docs` directory.\n\n## Release Management\n\nIn order to manage changelogs, version bumps and github releases, we use [cocogitto](https://github.com/cocogitto/cocogitto), or `cog` for short. Follow the [Installation instructions](https://github.com/cocogitto/cocogitto?tab=readme-ov-file#installation) to install `cog`. For Linux and Windows, we recommend using [cargo](https://doc.rust-lang.org/cargo/getting-started/installation.html) to install. For MacOS, we recommend using [brew](https://brew.sh/).\n\nThe main configuration file is `cog.toml`, which we run using some helper scripts located in the `tools/` directory.\n\nYou will also need the [GitHub CLI](https://cli.github.com/) tool installed to create and push releases to GitHub.\n\n### Publishing a New Plugin\n\nIf you've added a new plugin to the `plugins/` directory, there are a couple things you need to do to get it setup for releases and automatically bumping using `cog`.\n\n1. Add an entry for the new plugin in `cog.toml` under the `[package]` section. Use the existing entries as examples.\n2. Update the `tools/update_version.sh` script to include the new plugin and where its version is stored in source code. If your component has both a Python and JS component, you'll need to add it to both sections. Use the existing entries as examples.\n   - For an example, check the [PR that added the pivot plugin for release](https://github.com/deephaven/deephaven-plugins/pull/1242/files).\n3. Add the new plugin to the `.github/workflows/modified-plugin.yml` file so that a tag pushed with the name of the plugin will trigger the modified plugin workflow.\n4. In PyPI, Add a new pending publisher for the project.\n   1. Go to https://pypi.org/manage/account/publishing/\n   2. Under \"Add a new pending publisher\" section, enter the following:\n      - PyPI Project Name: The name of the python plugin, e.g. `deephaven-plugin-ui`\n      - Owner: `deephaven`\n      - Repository name: `deephaven-plugins`\n   3. Click \"Add\"\n5. For npmjs, the project should already be scoped to the `@deephaven` or `@deephaven-enterprise` organization. You may need to update the `package.json` to include the correct `repository` if it is not already set. Add the following to your `package.json`:\n   ```json\n   \"repository\": {\n     \"type\": \"git\",\n     \"url\": \"git+https://github.com/deephaven/deephaven-plugins.git\"\n   }\n   ```\n\nAfter completing the above, you should be able to run the release script in the [Cutting a New Release](#cutting-a-new-release) section. After the first release, verify that the version numbers are being updated correctly in source code, and that the releases are being created correctly on GitHub, PyPI, and npmjs. You should also update the project and add `deephaven` as an owner on the project in PyPI.\n\n### Cutting a New Release\n\nIn order to release a given plugin, you will run the script: `tools/release.sh \u003cpluginName\u003e`.  \nThis must be done on a branch named `main` and will publish to the `git remote -v` named `origin` by default.\n\nYou can specify a different remote using the `--remote` (or `-r`) flag:\n```bash\ntools/release.sh --remote upstream \u003cpluginName\u003e\n```\nThis is useful when your `origin` points to a fork and you want to release to the upstream repository.\n\n`tools/release.sh \u003cpluginName\u003e` will validate that your system has the necessary software installed and setup correctly, then invoke `cog bump --auto --package \u003cpluginName\u003e`,  \nwhich will invoke the necessary programs and scripts to automate a version bump and GitHub release.\n\nDuring development, it is expected that all commit message will adhere to [conventional commits]([https://www.conventionalcommits.org/en/about/]).\n`cog` will then uses your commit messages to compute a new version number, assemble a changelog, update our version in source code, create and push git tags, and perform a GitHub release for the given plugin.\n\nSee `cog.toml` to understand the full details of the release process.\n\nAfter you have successfully run `tools/release.sh` once, you should be able to directly invoke `cog bump --auto --package \u003cpluginName\u003e`, or omit the `--package` to release all plugins which have updated files.\n\n### Updating Versions in Source Code\n\nAs part of the release process, `cog` will, per our `cog.toml` configuration, invoke `tools/update_version.sh \u003cpackageName\u003e \u003cnewVersion\u003e`, which is a script that uses `sed` to update a plugin's version number in whatever source file we happen to use as the source of truth for version information in the given plugin.\n\n_[WARNING]_ If you change where the source of truth for a plugin's version is located, you must update `tools/update_version.sh` to update the correct file with a new version number.\n\nWe use `tools/update_version.sh` to remove any `.dev0` \"developer version\" suffix before creating a release, and to put the `.dev0` version suffix back after completing the release.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeephaven%2Fdeephaven-plugins","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdeephaven%2Fdeephaven-plugins","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeephaven%2Fdeephaven-plugins/lists"}