{"id":29200904,"url":"https://github.com/datopian/ckanext-querytool","last_synced_at":"2025-10-13T07:32:29.060Z","repository":{"id":37068562,"uuid":"111704659","full_name":"datopian/ckanext-querytool","owner":"datopian","description":":mag: :bar_chart: :earth_asia: CKAN extension that will provide data querying and story sharing with pre configured set of rules :newspaper: :clipboard:","archived":false,"fork":false,"pushed_at":"2025-05-22T11:43:02.000Z","size":16434,"stargazers_count":19,"open_issues_count":0,"forks_count":12,"subscribers_count":14,"default_branch":"master","last_synced_at":"2025-07-09T14:51:42.182Z","etag":null,"topics":["c3js","ckan","ckan-extension","ckanext","datatables","leafletjs","python","sqlalchemy"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/datopian.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2017-11-22T15:58:32.000Z","updated_at":"2025-05-22T11:43:05.000Z","dependencies_parsed_at":"2023-11-28T23:24:10.615Z","dependency_job_id":"f87b5579-83c8-4141-bfcf-187f7723d914","html_url":"https://github.com/datopian/ckanext-querytool","commit_stats":null,"previous_names":[],"tags_count":177,"template":false,"template_full_name":null,"purl":"pkg:github/datopian/ckanext-querytool","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datopian%2Fckanext-querytool","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datopian%2Fckanext-querytool/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datopian%2Fckanext-querytool/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datopian%2Fckanext-querytool/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/datopian","download_url":"https://codeload.github.com/datopian/ckanext-querytool/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datopian%2Fckanext-querytool/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279014119,"owners_count":26085463,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-13T02:00:06.723Z","response_time":61,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["c3js","ckan","ckan-extension","ckanext","datatables","leafletjs","python","sqlalchemy"],"created_at":"2025-07-02T11:06:15.210Z","updated_at":"2025-10-13T07:32:29.040Z","avatar_url":"https://github.com/datopian.png","language":"JavaScript","readme":"# ckanext-querytool\n\nA CKAN extension to create Plotly, DataTable, and Leaflet visualizations using dataset resources.\n\n\u003c!-- START doctoc generated TOC please keep comment here to allow auto update --\u003e\n\n**Table of Contents**  *generated with [DocToc](https://github.com/ktechhub/doctoc)*\n\n\u003c!---toc start--\u003e\n\n* [ckanext-querytool](#ckanext-querytool)\n  * [Creating new releases](#creating-new-releases)\n    * [Test data seed command](#test-data-seed-command)\n  * [Development Environment installation](#development-environment-installation)\n  * [Custom Development Within the ckanext-querytool Extension](#custom-development-within-the-ckanext-querytool-extension)\n    * [Frontend](#frontend)\n      * [Plotly](#plotly)\n      * [DataTables](#datatables)\n      * [Leaflet](#leaflet)\n      * [General](#general)\n      * [UI Options for Charts and Tables](#ui-options-for-charts-and-tables)\n      * [HTML/Jinja2 Templates](#htmljinja2-templates)\n      * [Language selector](#language-selector)\n    * [Backend](#backend)\n      * [API/Actions](#apiactions)\n      * [Helpers](#helpers)\n      * [Visualizations Object](#visualizations-object)\n  * [Updating README](#updating-readme)\n  * [Config Settings](#config-settings)\n    * [Optional Map Config Settings](#optional-map-config-settings)\n    * [Optional Cookie Control Config Settings](#optional-cookie-control-config-settings)\n    * [Optional reCAPTCHA v2 on login page](#optional-recaptcha-v2-on-login-page)\n* [Example values](#example-values)\n  * [Google Analytics (GA4)](#google-analytics-ga4)\n\n\u003c!---toc end--\u003e\n\n\u003c!-- END doctoc generated TOC please keep comment here to allow auto update --\u003e\n\n## Creating new releases\n\nBefore creating a new release, see the next section, [Test data seed command](#test-data-seed-command)\n\n### Test data seed command\n\nOn a freshly installed portal, with nothing but an admin user, you can run a command to populate the portal with test data found on the [STEPS Data](https://vital-stepsdata.org/) site. This includes the STEPS Survey organization, 9 datasets with 2 resources each, and 9 reports with multiple visualizations.\n\nBefore creating a new release, this command should be run to verify that there are no compatibility issues. This could be especially critical if there are any changes to the DB tables or structure.\n\nTo populate the portal, run:\n\n```bash\n% ckan -c \u003cPATH_TO_INI\u003e vs seed\n```\n\nIt will then prompt the user to enter the admin name and admin API key:\n\n```\nEnter the admin login name: \u003cADMIN_NAME\u003e\n\nEnter the admin API key: \u003cAPI_KEY\u003e\n```\n\nThat's it! You'll see output for each step (but the overall run time is very fast) and it will alert you if it runs into any issues.\n\nIf you have issues running the command successfully, you might need to clean and re-init the DB:\n\n\u003e**WARNING**: The following commands will completely clear the DB, including the admin user, password, and email address. If you've created anything on the portal, you'll need to do so again.\n\n```bash\n% ckan -c \u003cPATH_TO_INI\u003e db clean\n% ckan -c \u003cPATH_TO_INI\u003e db init\n```\n\nAfter you've recreated the admin user, run the seed command again, with the new user name and API key.\n\n## Development Environment installation\n\nSee the [Docker Compose setup for Vital Strategies CKAN](https://github.com/datopian/ckan-docker-vs-dev).\n\n## Custom Development Within the ckanext-querytool Extension\n\nThis extension diverges from CKAN core in a number of ways. The following sections cover the major areas of divergence and areas most commonly customized, along with high-level descriptions and links to the relevant code and external documentation.\n\nThe frontend libraries and frameworks used by this extension for charts and tables are:\n\n- [Plotly](https://plotly.com/javascript/) for charts (bar, line, pie, etc.)\n- [DataTables](https://datatables.net/) for tables\n- [Leaflet](https://leafletjs.com/) for maps\n\nThese are used on top of:\n\n- [jQuery](https://jquery.com/)\n- [Bootstrap](https://getbootstrap.com/)\n- [JavaScript](https://developer.mozilla.org/en-US/docs/Web/JavaScript)\n\nTo rebuild the JavaScript files, use [webpack](https://webpack.js.org/) (if you need to install it, you can do so with `npm install -g webpack`—currently, Node v10-v16 are known to work—**TODO**: update compatibility for newer a Node version):\n\n```bash\nnpm run webpack\n```\n\nFor more information on CKAN core architecture as a whole, see [CKAN code architecture](https://docs.ckan.org/en/2.7/contributing/architecture.html?highlight=helpers). For anything not covered here, see the [CKAN 2.10 core documentation](https://docs.ckan.org/en/2.10/).\n\n### Frontend\n\n#### Plotly\n\nThis extension uses [plotly.js](https://plotly.com/javascript/) for charts (bar, line, pie, etc.). It's a JavaScript library for creating interactive charts and graphs, built on top of [d3.js](https://d3js.org/) and [stack.gl](https://github.com/stackgl).\n\nMost changes and customizations of plotly.js are done in the `ckanext-querytool/ckanext/querytool/assets/javascript/modules/viz-preview.js` file.\n\nThe main settings object can be found in the [`viz-preview.js` file](ckanext/querytool/assets/javascript/modules/viz-preview.js), and it's called `base_info`. It contains the following properties:\n```\nvar base_info = {\n  margin: {\n    l: 20,\n    r: 20,\n    b: 20,\n    t: 30,\n    pad: 5,\n  },\n  title: titleVal,\n  showlegend: show_legend, //show legend value\n  legend: {\n    xanchor: \"left\",\n    x: -0.02,\n    y: -0.27,\n    orientation: \"h\",\n    title: {\n      text: legend_title_text,\n      side: \"top\",\n      //  Reference: https://plotly.com/javascript/reference/layout/#layout-legend\n      font: {\n        size: 15\n      }\n    }\n  },\n  xaxis: {\n    tickformat: x_tick_format,\n    automargin: true,\n    title: \"\",\n    tickangle: this.options.x_text_rotate,\n    tickmode: \"auto\",\n    nticks: this.options.x_tick_culling_max,\n    tickfont: {\n      size: 14,\n    },\n    categoryorder: \"array\",\n    categoryarray: sortedArr\n  },\n  yaxis: {\n    tickformat: y_tick_format,\n    automargin: true,\n    hoverformat: format,\n    tickangle: this.options.y_text_rotate,\n    tickfont: {\n      size: 14,\n    },\n  },\n  hovermode: \"closest\",\n};\n```\n\nAny of these properties can be modified to change the default behavior of the chart. For example, if you want to change the default font size of the chart, you can change the `size` property of the `tickfont` object. Additionally, each chart type has its own set of properties that can be modified by looking for the respective `trace` object. For example, each line chart type has the following properties in their `trace` objects:\n```\nvar trace = {\n  x: x,\n  y: columns[tmp].slice(1),\n  type: \"scatter\",\n  mode: labelsMode,\n  text: convertedTextTitles,\n  textposition: \"top right\",\n  textfont: {\n    size: 14,\n  },\n  name: name,\n  line: {\n    width: setDefaultWidth(lineWidthsList[tmp]),\n    dash: lineTypesList[tmp],\n  },\n  hovertemplate: \"%{y}\u003cextra\u003e\u003c/extra\u003e\",\n  error_y: {},\n  error_x: {},\n};\n```\n\n**NOTE:** Though all of the properties are customizable, it's recommended to only change them if absolutely necessary, as any changes can have unintended consequences.\n\nFor more information on how to customize plotly.js, see the [plotly.js documentation](https://plotly.com/javascript/).\n\n#### DataTables\n\nThis extension uses [DataTables](https://datatables.net/) for table charts/visualizations. DataTables is a JavaScript library for creating interactive tables, built on top of [jQuery](https://jquery.com/).\n\nChanges and customizations of DataTables can generally be handled in `ckanext-querytool/ckanext/querytool/assets/javascript/vitals.js` or `ckanext-querytool/ckanext/querytool/assets/javascript/modules/table-module.js`.\n\nAn example of custom changes can be found at the very top of [`vitals.js`](ckanext/querytool/assets/javascript/vitals.js), where a feature plugin (`RowsGroup`) is used to automatically merge columns cells based on their values eqaulity.\n\nFor more information on how to customize DataTables, see the [DataTables documentation](https://datatables.net/manual/).\n\n#### Leaflet\n\nThis extension uses [Leaflet](https://leafletjs.com/) for map charts/visualizations. Leaflet is a JavaScript library for creating interactive maps, built on top of [OpenStreetMap](https://www.openstreetmap.org/).\n\nChanges and customizations of Leaflet can generally be handled in `ckanext-querytool/ckanext/querytool/assets/javascript/modules/map-module.js`.\n\n#### General\n\nMost general jQuery and JavaScript customizations/overrides can be handled in `ckanext-querytool/ckanext/querytool/assets/javascript/vitals.js`. Here's a small example where we watch for changes in the filter dropdowns and trigger a click on the \"Update\" button when a change is detected:\n```\n%(\"body\").on('change','.has-filter .filter-item-value', function() {\n  %('.btn-update').trigger(\"click\");\n})\n```\n\nFor more information on how to customize jQuery and JavaScript, see the [jQuery documentation](https://api.jquery.com/) and [JavaScript documentation](https://developer.mozilla.org/en-US/docs/Web/JavaScript).\n\n#### UI Options for Charts and Tables\n\nFor most changes to the options found in the UI (for charts and tables, e.g. https://YOUR_SITE/report/edit_visualizations/REPORT_NAME), you can find the relevant code in the following files:\n\n- [viz-preview.js](ckanext/querytool/assets/javascript/modules/viz-preview.js)\n- [visualizations_settings.js](ckanext/querytool/assets/javascript/visualizations_settings.js)\n\n#### HTML/Jinja2 Templates\n\nCKAN (and this extension) uses [Jinja2](https://jinja.palletsprojects.com/en/3.1.x/) for templating. The templates for this extension can be found in [`ckanext-querytool/ckanext/querytool/templates`](ckanext/querytool/templates). Any changes in these HTML files should be reflected immediately in the UI immediately, but if you're not seeing them, you may need to restart CKAN. \n\nAdditionally, if you need to override a template found in CKAN core, you can simply copy the file found in CKAN core into the `ckanext-querytool/ckanext/querytool/templates` directory and make your changes there. You must make sure that it's placed in the same directory structure as the original file, e.g. if you want to override the `package/read.html` template, you would copy the file from `ckan/templates/package/read.html` to `ckanext-querytool/ckanext/querytool/templates/package/read.html`.\n\nFor more information on how to customize Jinja2 templates, see the [Jinja2 documentation](https://jinja.palletsprojects.com/en/3.1.x/), [CKAN Jinja2 documentation](https://docs.ckan.org/en/2.10/theming/templates.html#jinja2), and [CKAN theming documentation](https://docs.ckan.org/en/2.10/theming/templates.html#replacing-a-default-template-file).\n\n#### Language selector\n\nFor the country flags to appear in the language selector it's needed to make sure there's an associated flag for each one of the configured languages. This must be done by further expanding the `/ckanext-querytool/ckanext/querytool/public/base/resources/lang-flags.json` file. Here's an example of what this file looks like:\n```\n{\n    \"en\": \"gb\",\n    \"pt_BR\": \"br\",\n    \"es\": \"es\",\n    \"fr\": \"fr\",\n    \"km\": \"kh\",\n    \"zh_CN\": \"cn\"\n}\n```\nThe property name is always the language value. The value is always the ISO 3166-1-alpha-2 code of the country.\n\n### Backend\n\n#### API/Actions\n\nThe process for creating/customizing the API endpoints (Actions) is the slightly different than the [CKAN documentation](https://docs.ckan.org/en/2.10/extensions/index.html), but this is only because we have automated `get_actions` in `plugin.py` to return all actions found in [`ckanext-querytool/ckanext/querytool/logic/action`](ckanext/querytool/logic/action) (in this directory, we have 4 files: `create.py`, `delete.py`, `get.py`, and `update.py`, all of which are included in the automated loading), so they don't need to be added individually. We do this with the following code (found in [`plugin.py`](ckanext/querytool/plugin.py):\n```\nmodule_root = 'ckanext.querytool.logic.action'\naction_functions = h._get_functions(module_root)\n```\n\nAs a basic example, let's say we want to create a new GET endpoint that simply returns the text \"This is our new API endpoint!\".\n\n1. We would simply add the following code to [`ckanext-querytool/ckanext/querytool/logic/action/get.py`](querytool/logic/action/get.py):\n    ```\n    @toolkit.side_effect_free\n    def our_new_action(context, data_dict):\n        return 'This is our new API endpoint!'\n    ```\n2. Restart CKAN\n3. Visit https://YOUR_SITE/api/3/action/our_new_action, and you should see the following response:\n    ```\n    {\n      \"help\": \"https://YOUR_SITE/api/3/action/help_show?name=our_new_action\",\n      \"success\": true,\n      \"result\": \"This is our new API endpoint!\"\n    }\n    ```\n\n#### Helpers\n\nHelpers are re-usable functions. They're available in the backend Python code as well as in Jinja2 templates. There's basically no limit to what a helper can do. They can range from simple functions that return a string, to more complex functions that parse, filter/alter, and return data.\n\nThe process for creating/customizing helpers is the same as found in the [CKAN helper documentation](https://docs.ckan.org/en/2.10/theming/templates.html#adding-your-own-template-helper-functions).\n\nAs a basic example, let's say we want to create a new helper that simply returns the text \"This is our new helper!\".\n\n1. We would start by adding the following code to [`ckanext-querytool/ckanext/querytool/helpers.py`](ckanext/querytool/helpers.py):\n    ```\n    def our_new_helper():\n        return 'This is our new helper!'\n    ```\n2. Add our new helper to `get_helpers` in [`plugin.py`](ckanext/querytool/plugin.py):\n    ```\n    def get_helpers(self):\n        return {\n            ... OTHER HELPERS ...\n            'our_new_helper': helpers.our_new_helper\n        }\n    ```\n3. Call the new helper in another function or in a template. For example, in a template, we can call it with:\n    ```\n    {{ h.our_new_helper() }}\n    ```\n4. Restart CKAN\n\nNow, you should see the text \"This is our new helper!\" in the template wherever you called it (or you can add a log in the backend function you called it from to see the output in the logs).\n\nFor a better idea of what helpers can be used for, you can take a look at the helpers in [`ckanext-querytool/ckanext/querytool/helpers.py`](ckanext/querytool/helpers.py) and [CKAN core helpers](https://github.com/ckan/ckan/blob/ckan-2.10.4/ckan/lib/helpers.py).\n\n#### Visualizations Object\n\nIf you need to intercept or add new keys/values to the backend visualizations or reports objects, you can find much of the code in [`/ckanext/querytool/controllers/querytool.py`](ckanext/querytool/controllers/querytool.py) (for example, `edit_visualizations` retrieves the saved values from the DB and returns them to the template) and [`/ckanext/querytool/model.py`](ckanext/querytool/model.py) (this is where new DB columns are added).\n\n## Updating README\n\nTo update this README's table of contents, use `doctoc` (if you don't have it installed already, you can install in various ways, such as `npm install -g doctoc` or `pip install --user doctoc`).\n\n```bash\ndoctoc README.md\n```\n\n## Config Settings\n\nThe extension supports some optional configurations:\n\n- OpenStreetMaps\n- Cookie Control\n- reCAPTCHA v2 on login page\n\n### Optional Map Config Settings\n\nThe example values below are the defaults. To override them, add the following to your `.ini` or `.env` file and change the values as needed:\n\n```\nckanext.querytool.map_osm_url = https://cartodb-basemaps-{s}.global.ssl.fastly.net/light_nolabels/{z}/{x}/{y}{r}.png\nckanext.querytool.map_osm_attribute = \u0026copy; \u003ca href=\"http://www.openstreetmap.org/copyright\"\u003eOpenStreetMap\u003c/a\u003e \u0026copy; \u003ca href=\"http://cartodb.com/attributions\"\u003eCartoDB\u003c/a\u003e\n```\n\nFor example, if you want to use a dark CartoDB basemap instead of the default light map, you can set the variables to the following:\n\n```\nckanext.querytool.map_osm_url = https://cartodb-basemaps-{s}.global.ssl.fastly.net/dark_nolabels/{z}/{x}/{y}{r}.png\n```\n\nThe second variable is used for the attribution/copyright text. For example, if you decide to use OpenStreetMap without CartoDB, you can set the variables to the following:\n\n```\nckanext.querytool.map_osm_url = https://tile.openstreetmap.org/{z}/{x}/{y}.png\nckanext.querytool.map_osm_attribute = \u0026copy; \u003ca href=\"http://www.openstreetmap.org/copyright\"\u003eOpenStreetMap\u003c/a\u003e\n```\n\n**NOTE:** Changing the map provider, though possible, is not recommended. If the alternative isn't a complete and identical drop-in replacement, the map will most likely **not** work as expected, and you will need to make changes to the code to accommodate the new map provider.\n\n### Optional Cookie Control Config Settings\n\nThe [Cookie Control](https://www.civicuk.com/cookie-control/) panel can be configured using the following options:\n\n```\nckanext.querytool.cc.enabled = True\nckanext.querytool.cc.api_key = your_api_key\nckanext.querytool.cc.license_type = COMMUNITY\nckanext.querytool.cc.popup_position = LEFT # Or RIGHT\nckanext.querytool.cc.theme_color = DARK    # Or LIGHT\nckanext.querytool.cc.initial_state = OPEN  # Or CLOSED\n```\n\nBy default, if `enabled` is true, only `api_key` is required for the Cookie Control to work. Currently, it considers `auth_tkt` as a core cookie and Google Analytics Cookies as optionals. An API key can be generated by [registering here](https://www.civicuk.com/cookie-control/).\n\n### Optional reCAPTCHA v2 on login page\n\nIt's possible o enable reCAPTCHA v2 on the login page by setting the following environment variables:\n\n```bash\n# Example values\nckanext.querytool.recaptcha.sitekey = 6LeIxAcTAAAAAJcZVRqyHh71UMIEGNQ_MXjiZKhI\nckanext.querytool.recaptcha.secretkey = 6LeIxAcTAAAAAGG-vFI1TnRWxMZNFuojJ4WifJWe\n```\n\n## Google Analytics (GA4)\n\nTo setup Google Analytics using GA4, you need to install a specific branch (`ed-dev-ga4`) of the Datopian Google Analytics extension:\n\n[`ckanext-googleanalytics` with GA4 support](https://github.com/datopian/ckanext-googleanalytics/tree/ed-dev-ga4)\n\nThen follow the instructions in the [\"Setup\" section of the README](https://github.com/datopian/ckanext-googleanalytics/tree/ed-dev-ga4#setup).\n\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatopian%2Fckanext-querytool","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdatopian%2Fckanext-querytool","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatopian%2Fckanext-querytool/lists"}