{"id":19721987,"url":"https://github.com/datajoint/sci-viz","last_synced_at":"2025-04-29T22:30:32.805Z","repository":{"id":38444140,"uuid":"407662444","full_name":"datajoint/sci-viz","owner":"datajoint","description":"Generic visualization framework for building dashboarding capabilities for DataJoint pipelines.","archived":false,"fork":false,"pushed_at":"2023-11-06T21:01:50.000Z","size":3410,"stargazers_count":6,"open_issues_count":26,"forks_count":12,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-04-05T19:51:15.513Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://datajoint.com/docs/core/sci-viz","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/datajoint.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2021-09-17T19:49:19.000Z","updated_at":"2023-06-28T17:48:49.000Z","dependencies_parsed_at":"2023-11-07T03:10:28.168Z","dependency_job_id":"b6dcb45c-ad47-4629-88c9-3fa5b41d85d3","html_url":"https://github.com/datajoint/sci-viz","commit_stats":null,"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datajoint%2Fsci-viz","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datajoint%2Fsci-viz/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datajoint%2Fsci-viz/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datajoint%2Fsci-viz/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/datajoint","download_url":"https://codeload.github.com/datajoint/sci-viz/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251592898,"owners_count":21614441,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-11T23:16:08.741Z","updated_at":"2025-04-29T22:30:32.346Z","avatar_url":"https://github.com/datajoint.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# DataJoint SciViz\n\n- DataJoint SciViz is a generic scientific visualization framework for building dashboards of DataJoint pipelines.\n\n## References\n\n- [DataJoint Documentation](https://datajoint.com/docs)\n- [DataJoint SciViz Documentation](https://datajoint.com/docs/core/sci-viz)\n- [Pharus - DataJoint REST API backend](https://github.com/datajoint/pharus)\n\n## Installation\n\nIf you have not done so already, please install the following dependencies:\n\n- [Docker](https://docs.docker.com/get-docker/)\n- [Docker Compose](https://docs.docker.com/compose/install/)\n\n## Prerequisites\n\nBefore running the application through docker, make sure to run the command:\n\n```bash\npython frontend_gen.py\n```\n\nThis will compile the necessary typescript files for the frontend and the necessary python files for the backend.\nyou can also run this at any time during a dev deployment to hot-load the frontend, however this is not always guaranteed to work as some changes may require the entire react server to restart. Anything related to the back end api is not hot-loadable but for example the component locations and sizes can be modified and hot-loaded.\n\nAlso see the _.env_ section to set up your environment variables.\n\n## Running the application\n\nTo start the application in dev mode, use the command:\n\n```bash\nPY_VER=3.8 IMAGE=djbase DISTRO=alpine PHARUS_VERSION=$(cat pharus/pharus/version.py | tail -1 | awk -F\\' '{print $2}') HOST_UID=$(id -u) docker-compose up --build\n```\n\nTo run the application in Production mode, use the command:\n\n```bash\ndocker-compose -f docker-compose-deploy.yaml up --build\n```\n\nTo stop the application, use the same command as before but with `down` in place of `up --build`\n\n## Dynamic spec sheet\n\nSci-Viz is used to build visualization dashboards, this is done through a single spec sheet. An example spec sheet is included named `example_visualization_spec.yaml`\n\n\u003cdetails\u003e\n\u003csummary\u003eClick to expand details\u003c/summary\u003e\n\nImportant notes about restrictions in the spec sheet:\n\n- Page names under pages must have a unique name without spaces\n- Page routes must be unique\n- Grid names under grids must be unique without spaces\n- Component names under components must be unique **but can have spaces**\n- The routes of individual components must be unique\n- Routes must start with a `/`\n- Every query needs a restriction, below is the default one.\n  - ```python\n        def restriction(**kwargs):\n            return dict(**kwargs)\n    ```\n- Overlapping components at the same (x, y) does not work, the grid system will not allow overlapping components it will wrap them horizontally if there is enough space or bump them down to the next row.\n\nIf the website does not work after running the frontend generation script check this list to make sure that spec sheet is constructed properly, in the future we may include a script that lints the spec sheet for you. see issue [#20](https://github.com/datajoint/sci-viz/issues/20)\n\n\u003c/details\u003e\n\n## Grids\n\nSci-Viz produces custom visualizations by putting `grids` on `pages` and then filling them with visualization `components`. Currently there are two types of grids **Fixed** and **Dynamic**\n\n\u003cdetails\u003e\n\u003csummary\u003eClick to expand details\u003c/summary\u003e\n\n### Fixed mode grid\n\nA fixed mode grid requires all components to explicitly give their position and size on the grid.\n\nA fixed `grid` takes 4 arguments:\n\n- `type:` indicates the type of grid, in this case `type: fixed`\n- `columns:` the number of columns that the grid will have\n- `row_height:` the height of each row in pixels\n- `components:` a yaml dictionary of components to be spawned in the grid\n\n### Dynamic grid mode\n\nA dynamic grid takes a datajoint query and then uses each record and applies that record as a restriction to a template of components. It then spawns a single or group of components for each record of that parent query but the components query is restricted by the entire record that has been passed in from the parent query.\n\nAn example of this would be as follows:\n\n- You have one table that represents all identifying data of a subject, lets use Mouse as an example for the subject and the table name\n- You also have a table that contains a single plot per Mouse primary key, lets call this table MousePlots\n- You have no idea how many plots are in MousePlots but you want to display a live view of all of them\n- What you can do is create a dynamic grid with the parent query being for the Mouse table and a plot component with a query for the MousePlot table. This will produce all of the plots that are available without knowing how many there are in the database.\n\nA dynamic `grid` takes 7 arguments:\n\n- `type:` indicates the type of grid, in this case `type: fixed`\n- `columns:` the number of columns that the grid will have\n- `row_height:` the height of each row in pixels\n- `restriction:` a restriction for the datajoint query\n- `dj_query:` the parent datajoint query that will provide the restriction records\n- `route:` backend api route for the parent query\n- `component_templates:` a yaml dictionary of components that serve as a template\n\nAdditionally any components in the dynamic grid do not need `x`, `y` , `height`, and `width` fields.\n\nCurrently only the `metadata` and `plot` components are supported in dynamic mode.\n\n\u003c/details\u003e\n\n## Components\n\n\u003cdetails\u003e\n\u003csummary\u003eClick to expand details\u003c/summary\u003e\n\nAll components need minimally these fields:\n\n- `type:` indicates the type of component you are trying to generate\n- `x:` x position on the grid starting at 0\n- `y:` y position on the grid starting at 0\n- `height:` the amount of grid squares tall a component can be, minimum 1\n- `width:` the amount of grid square wide a compoentnt can be, minimum 1\n\n### Table component\n\n`type:` table\n\nThe Table component takes a few additional fields:\n\n- `route:` the backend route for the rest api query, must start with a `/`\n- `restriction:` the restriction for the datajoint query\n- `dj_query:` the datajoint query for for the table data\n\nIf setup correctly the component will render the result of the query in a table that supports paging, sorting, and filtering.\n\n#### Adding color to your tables using projections\n\n```python\ndef dj_query(vms):\n    TableA, TableB = (vms['test_group1_simple'].TableA, vms['test_group1_simple'].TableB)\n    return ((TableA * TableB).proj(...,\n                                   _sciviz_font='IF(a_name = \"Raphael\", \"rgb(255, 0, 0)\", NULL)',\n                                   _sciviz_background='IF(a_name = \"Raphael\", \"rgba(50, 255, 0, 0.16)\", NULL)',)\n                                  ), dict(order_by='b_number')\n```\n\nThis is an example of a table query that has a projection that applys a text color as well as a background color.\nIt does so through the use of 2 protected column names:\n\n- `_sciviz_font` for the font color\n- `_sciviz_background` for the background color\n  these two fields will accept any color format that css does.\n\nIn the example we do a join of two tables and then do a projection where we create 2 new columns with the protected names and if a condition is met we set their field to a css-compatable color else we have it be `NULL`. In the example above we use rgb when we do not need transparency and rgba when we do.\n[here is a good tool for picking css colors.](https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Colors/Color_picker_tool)\n\n### Markdown component\n\n`type:` markdown\n\nThe markdown component takes one additional field `text: |`\nunderneath the `|` operator you can place any markdown text block that you want.\n\n### Plot component from stored Plotly JSON\n\n`type:` plot:plotly:stored_json\n\nThe plot component takes 3 additional arguments:\n\n- `route:` the backend route for the rest api query, must start with a `/`\n- `restriction:` the restriction for the datajoint query\n- `dj_query:` the datajoint query for for the table data.\n\nThe plot component also takes one optional argument:\n\n- `channels:` (string Arr) channels to listen to for additional restrictions from other components (slider, dropdown, ect.)\n\nAdditionally for the plot to render properly the result of your query must be a single entry with one element that is a plotly JSON.\nAn easy way to do this is to set the `fetch_args=[]` in your `dj_query` to be only the column that contains a plotly JSON and additionaly set your restriction to be the index of the plot you are looking for\n\n### Metadata component\n\n`type:` metadata\n\nThe Metadata component takes 3 additional arguments:\n\n- `route:` the backend route for the rest api query, must start with a `/`\n- `restriction:` the restriction for the datajoint query\n- `dj_query:` the datajoint query for for the table data.\n\nAdditionally the metadata component only takes a single row from a table as its input so the `dj_query` and `restriction` need to be properly set to produce a single record. This component is not very useful by itself but when combined with other components as part of a template in a `Dynamic grid` it can provide useful information on what the other components are showing.\n\n### Image component\n\n`type:` file:image:attach\n\nthe Image component takes 3 additional arguments:\n\n- `route:` the backend route for the rest api query, must start with a `/`\n- `restriction:` the restriction for the datajoint query\n- `dj_query:` the datajoint query for for the table data.\n\nAdditionally the image that you want to display needs to be stored as a datajoint [attach](https://docs.datajoint.org/python/definition/06.5-External-Data.html?highlight=attach) attribute type and your query should produce only one record with one column which is the column where the image is stored.\n\n### Form component\n\n`type:` form\n\nThe Form component takes 2 additional arguments:\n\n- `route:` the backend route for the rest api query, must start with a `/`\n- `tables:` the list of tables in \"`schema.table`\" format to insert into\n  - Table names can be templated, either fully or partially, using the `'{keyword}'` format. This keyword can then be assigned a value by an emitter component as a query parameter.\n\nThe Form can also take 2 optional argument:\n\n- `map:` a mapping to change the displayed names of the fields in the form\n\n  A map takes a _list_ of 3 arguments:\n\n  - `type:` attribute | table\n  - `input:` the new name of the field\n  - `destination:` the field to be renamed\n\n  A map entry with a `table` type can also take 1 optional argument:\n\n  - `map:` a nested mapping of the same structure to change the displayed names of the table's primary key attributes\n\n- `channels:` an array of channels to listen to for templated table name values.\n\n### Slider component\n\nThe slider is a component that takes a datajoint query and creates a slider based off the payload that the query returns. It turns each record into an index on the slider and also emits the currently selected record on its channel as a restriction to other components.\n\n`type:` slider\n\nthe Slider component takes 3 additional arguments:\n\n- `route:` the backend route for the rest api query, must start with a `/`.\n- `restriction:` the restriction for the datajoint query.\n- `dj_query:` the datajoint query for for the table data.\n- `channel:` the name of the channel that the slider outputs its restriction on.\n\nThe Slider also takes one optional argument:\n\n- `channels:` the array of channels to listen to for restricting its own query.\n\n### Radiobutton/dropdown-static component\n\nSimilar to the Slider, the radiobutton and dropdown-static components are components that supply a selected restriction on a channel to a component that can accept them.\n\n`type:` radiobuttons | dropdown-static\n\nthe Radiobutton/dropdown-static component takes 2 additional arguments:\n\n- `channel:` the name of the channel that the Radiobutton/dropdown-static outputs its restriction on.\n- `content:` dictionary of key value pairs, the key is what text is shown to the user while the value is the actual restriction. example:\n\n```\ncontent:\n  mouse 0: 'mouse_id=0'\n  mouse 1: 'mouse_id=1'\n  mouse 2: 'mouse_id=2'\n```\n\n### dropdown-query component\n\nThe dropdown-query component is the same as the slider component except it only expects a result with one column.\n\n`type:` dropdown-query\n\nthe dropdown-query component takes 3 additional arguments:\n\n- `route:` the backend route for the rest api query, must start with a `/`.\n- `restriction:` the restriction for the datajoint query.\n- `dj_query:` the datajoint query for for the table data.\n- `channel:` the name of the channel that the slider outputs its restriction on.\n\n\u003c/details\u003e\n\n## Developer instructions\n\nThere are a couple issues to address if you are collaborating on this project\n\n- devs will have have to point the submodule to their own fork of pharus if they need to edit pharus to support new features for sci-viz.\n- That change to pharus would need to be pr'd and then merged into pharus before we can pr and merge their change to sci-viz as we probably dont want unreviewed code linked to sci-viz nor do we want the submodule pointing to their fork of pharus.\n\n### .env\n\nfor running frontend_gen.py you need this variable in your .env file\n\n```bash\nDJSCIVIZ_SPEC_PATH=test/test_spec.yaml\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatajoint%2Fsci-viz","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdatajoint%2Fsci-viz","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatajoint%2Fsci-viz/lists"}