{"id":18628044,"url":"https://github.com/jborrow/pageplot","last_synced_at":"2025-08-23T05:11:23.596Z","repository":{"id":82399539,"uuid":"402873822","full_name":"JBorrow/pageplot","owner":"JBorrow","description":"PagePlot is a no-code plotting system for rapid development of numerical models","archived":false,"fork":false,"pushed_at":"2022-02-07T21:24:21.000Z","size":1144,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-08-08T18:04:13.173Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/JBorrow.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-09-03T19:09:20.000Z","updated_at":"2021-11-19T15:23:02.000Z","dependencies_parsed_at":"2023-06-15T12:31:03.263Z","dependency_job_id":null,"html_url":"https://github.com/JBorrow/pageplot","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/JBorrow/pageplot","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JBorrow%2Fpageplot","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JBorrow%2Fpageplot/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JBorrow%2Fpageplot/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JBorrow%2Fpageplot/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/JBorrow","download_url":"https://codeload.github.com/JBorrow/pageplot/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JBorrow%2Fpageplot/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":271745491,"owners_count":24813503,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-23T02:00:09.327Z","response_time":69,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-07T04:44:56.181Z","updated_at":"2025-08-23T05:11:23.545Z","avatar_url":"https://github.com/JBorrow.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"Page Plot\n=========\n\nOne very useful thing to have when developing numerical models is an ability\nto rapidly generate a series of diagnostic figures. Usually, you will want\nthe same number and type of figures each and every time, which usually come\ndown to a series of scatter plots (with and without binning lines) alongside\nsome additional data from external sources.\n\nThere are a number of ways to do this; you can maintain a set of scripts to\ncreate each plot, or use a dashboard system (like\n[Plotly Dash](https://dash.plotly.com/)). Both of these come along with\ndownsides; for the individual scripts, a lot of time can be spent maintaining\nshared code that is frequently copied and pasted between scripts. Those\nscripts are essentially just boilerplate, with lots of calls to matplotlib's\nAPI, or a series of pandas calls. Dashboards are tricky in a HPC environment\nas they require hosting, and as such are significantly less portable.\n\nPagePlot attempts to slot in-between these solutions. It is a low-code\nsolution, with the code being used to describe the data format to PagePlot,\nwith no-code plotting. The figures can then be used to create a static,\nportable, dashboard.\n\nPagePlot attempts to succeed the successful\n[swift-pipeline](https://github.com/swiftsim/pipeline) project and\ngeneralise it for use throughout the community.\n\n\nCodeless Plotting\n-----------------\n\nA typical diagnostic plot may include a scatter of two variables, along\nside a median line to demonstrate the overall trend. Let's take a look at how\nthat would be done in a typical python script:\n\n```python\n\nimport matploylib.pyplot as plt\nimport numpy as np\nimport h5py\n\nx_low, x_high = [1e7, 1e10] # cm / s\ny_low, y_high = [1e3, 1e9] # km\nn_bins = 24\n\nwith h5py.File(\"my_file.hdf5\", \"r\") as handle:\n    x_data = handle[\"/My/X/Data\"][:] * 1e5 # km / s to cm / s\n    y_data = handle[\"/My/Y/Data\"][:] # km\n\n\nbins = np.logspace(np.log10(x_low), np.log10(x_high), n_bins)\n\n# ... Your favourite code that does median binning, but produces\nbin_centers = ...\nmedians = ...\nscatters = ...\n\nfig, ax = plt.subplots()\n\nax.set_xscale(\"log\")\nax.set_yscale(\"log\")\n\nax.scatter(x_data, y_data)\n\nax.errorbar(bin_centers, medians, scatters)\n\nax.set_xlim(x_low, x_high)\nax.set_ylim(y_low, y_high)\n\nax.set_xlabel(\"X quantity [cm / s]\")\nax.set_ylabel(\"Y quantity [km]\")\n\nfig.savefig(\"test.png\")\n```\n\nSeems pretty simple, right? All is well when this is one script, dealing with\na fixed data source, with consistent units. The problems originate when you\nhave changing data (what happens to this script when the simulation software\nstarts outputting the `x_data` in `cm / s`?), multiple people with different\nideas on how median lines (or other intermediate products) should be created\n(should the bin center be placed at the median `x` value or at the midpoint\nof the bin edges?), the data should be displayed and visualised\n(shaded regions, v.s. error bars, and so-on), all with different packages\nused to process the data. This rapidly becomes a big mess (if you're looking\nat this package - you've probably been there).\n\nThis code doesn't even need to exist. Here, we're just specifying that you would\nlike a plot of `x` against `y`. I could just as easily have specified this as:\n\n```json\n{\n    \"test\": {\n        \"x\": \"/My/X/Data\",\n        \"y\": \"/My/Y/Data\",\n        \"x_units\": \"cm/s\",\n        \"y_units\": \"km\",\n        \"median_line\": {\n            \"limits\": [\"1e7 cm/s\", \"1e10 cm/s\"],\n            \"bins\": 24\n        },\n        \"axes_limits\": {\n            \"limits_x\": [\"1e7 cm/s\", \"1e10 cm/s\"],\n            \"limits_y\": [\"1e3 km\", \"1e9 km\"]\n        },\n        \"scale_axes\": {\n            \"scale_x\": \"log\",\n            \"scale_y\": \"log\"\n        }\n    }\n}\n```\n\nThis contains the exact same amount of information (barring the internal data\nunits from the file, but those should be stored as metadata in the file\nanyway). It's much shorter, and is importantly implementation-independent.\nI could use this JSON to pass to a `R` script that understands it, or even\nwrite some intermediary to allow `ggplot` to understand it. This JSON\n(very convienently) happens to have the exact format used in `PagePlot`.\nThe library can then control the way that data is processed, and the styling\nand output options for the figures in a fully consistent way.\n\n\nInterfacing With Page Plot\n--------------------------\n\nPage Plot is built out of three major components:\n\n1. A data object that conforms to the `IOSpecification`. This effectively needs\n   a way of taking your chosen file and loading a relevant one dimensional array\n   (with included units using `unyt`) from a string. The string is what is given\n   to the `x`, `y`, and `z` parameters in the JSON.\n2. A series of extensions (built in, and external) that conform to the\n   `PlotExtension` specification. These can compute derived data, interact\n   directly with the `matplotlib.Axes` and `matplotlib.Figure` objects, and\n   even with the filesystem. The attributes of the `PlotExtension` objects\n   correspond directly to the JSON attributes in the configuration files.\n3. Glue that holds all of these pieces together, and even allows for the\n   production of webpages that allow for all figures to be displayed alongside\n   each other as a summary page.\n\n\nAbout Those Webpages\n--------------------\n\nOne of the built in extensions in `PagePlot` is called `metadata`, and has the\nspecification:\n\n```json\n\"metadata\" {\n    \"comment\": \"Descriptive comment, saved out to serialized data\",\n    \"title\": \"Plot title, for use on webpage\",\n    \"caption\": \"Wepbage caption for plot\",\n    \"section\": \"Section on webpage to use\",\n}\n```\n\nThis metadata is primarily used to populate the webpages that `PagePlot` is\nable to produce (hence the name `PagePlot`). These pages are split into\nsections, with each figure in each section having a title and caption. The\npages are made out of a simple, static, HTML in a single page, so sharing the\nfolder of plot files and the `index.html` is enough to fully share a\ndiagnostic output. \n\n\nPutting it all Together\n-----------------------\n\nRunning `PagePlot` is very simple. At the moment, as the API is a little in flux,\nthere's no specific command line tool (but there will be one in the future)!\n\nThe script below takes in a data filename from the first argument, and runs all\nof the `PagePlot` infrastructure.\n\n```python\n\"\"\"\nRuns PagePlot.\n\"\"\"\n\nfrom pathlib import Path\nfrom pageplot.runner import PagePlotRunner\nfrom pageplot.io.areposubfind import IOAREPOSubFind\nimport sys\n\ndata_filename = Path(sys.argv[1])\n\nconfig_loc = Path(\"/Global/Path/To/Config\")\nconfig_files = list(config_loc.glob(\"*.json\"))\nglobal_config_file = config_loc / Path(\"config.json\")\n\nconfig_files.remove(global_config_file)\n\nrunner = PagePlotRunner(\n    config_filename=global_config_file,\n    data=IOAREPOSubFind(filename=data_filename),\n    plot_filenames=config_files,\n)\n\nrunner.create_figures()\nrunner.create_webpage()\n```\n\nIn `/Global/Path/To/Config/black_holes.json`, we have:\n```json\n{   \n    \"stellar_mass_black_hole_mass\": {\n        \"two_dimensional_histogram\": {\n            \"limits_x\": [\"1e7 Solar_Mass\", \"1e11 Solar_Mass\"],\n            \"limits_y\": [\"1e5 Solar_Mass\", \"1e9 Solar_Mass\"],\n            \"spacing_x\": \"log\",\n            \"spacing_y\": \"log\",\n            \"bins\": 128,\n            \"norm\": \"log\"\n        },\n        \"median_line\": {\n            \"limits\": [\"1e7 Solar_Mass\", \"1e11 Solar_Mass\"],\n            \"display_as\": \"shaded\",\n            \"spacing\": \"log\",\n            \"bins\": 20\n        },\n        \"scale_axes\": {\n            \"scale_x\": \"log\",\n            \"scale_y\": \"log\"\n        },\n        \"axes_limits\": {\n            \"limits_x\": [\"1e7 Solar_Mass\", \"1e11 Solar_Mass\"],\n            \"limits_y\": [\"1e5 Solar_Mass\", \"1e9 Solar_Mass\"]\n        },\n        \"legend\": {},\n        \"x\": \"Subhalo/SubhaloMassInRadType[:, 4]\",\n        \"y\": \"Subhalo/SubhaloBHMass\",\n        \"x_units\": \"Solar_Mass\",\n        \"y_units\": \"Solar_Mass\",\n        \"metadata\": {\n            \"caption\": \"Stellar Mass-Black Hole Mass relation. The black hole mass here gives the sum of all of the black hole masses in the halo, rather than the mass of the SMBH.\",\n            \"section\": \"Black Holes\",\n            \"title\": \"Stellar Mass-Black Hole Mass Relation\"\n        }\n    }\n}\n\n```\nAs well as a small config file `config.json`,\n```json\n{\n    \"stylesheet\": \"/Global/Path/To/Config/mnras.mplstyle\"\n}\n```\nThere are other plot specification files included, too, but they're omitted\nhere for brevity.\n\nOnce ran on an appropriate (Illustris-TNG) data file, for which there\nis a built in `IOSpecification` called `IOAREPOSubFind`, this produces\na webpage that looks like:\n![Example PagePlot output](example_screenshot.png)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjborrow%2Fpageplot","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjborrow%2Fpageplot","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjborrow%2Fpageplot/lists"}