{"id":31763771,"url":"https://github.com/engisalor/quartz","last_synced_at":"2025-10-09T23:25:47.030Z","repository":{"id":224540124,"uuid":"591644012","full_name":"engisalor/quartz","owner":"engisalor","description":"An app for visualizing Sketch Engine API data","archived":false,"fork":false,"pushed_at":"2024-03-01T11:25:56.000Z","size":732,"stargazers_count":1,"open_issues_count":2,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-09-04T19:28:32.935Z","etag":null,"topics":["app-template","corpus-linguistics","dash-plotly","data-visualization","docker","python","sketch-engine"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/engisalor.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-01-21T11:50:07.000Z","updated_at":"2024-06-18T09:40:05.000Z","dependencies_parsed_at":"2024-03-01T12:42:24.684Z","dependency_job_id":null,"html_url":"https://github.com/engisalor/quartz","commit_stats":null,"previous_names":["engisalor/quartz"],"tags_count":5,"template":false,"template_full_name":null,"purl":"pkg:github/engisalor/quartz","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/engisalor%2Fquartz","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/engisalor%2Fquartz/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/engisalor%2Fquartz/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/engisalor%2Fquartz/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/engisalor","download_url":"https://codeload.github.com/engisalor/quartz/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/engisalor%2Fquartz/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279002322,"owners_count":26083340,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-09T02:00:07.460Z","response_time":59,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["app-template","corpus-linguistics","dash-plotly","data-visualization","docker","python","sketch-engine"],"created_at":"2025-10-09T23:25:45.279Z","updated_at":"2025-10-09T23:25:47.019Z","avatar_url":"https://github.com/engisalor.png","language":"Python","readme":"# Quartz\n\nQuartz is a container app template for visualizing corpus data from Sketch Engine servers. It's a portable alternate interface that focuses on graphing quantitative data for linguistic analysis. Set up access to your corpora, make queries via API requests to a Sketch Engine or NoSketch Engine server, and view results with interactive graphs.\n\nSome default graphing features are included, but the repo is designed for adaptation to specific projects. Quartz is made with Python, the Dash framework and Docker. To use it you'll need API access to a Sketch Engine or NoSketch Engine server.\n\n## Installation\n\nPython dependencies:\n\n`dash dash-bootstrap-components flask gunicorn plotly sgex`\n\nRelated software:\n\n- [Sketch Engine](https://www.sketchengine.eu/)\n- [NoSketch Engine](https://nlp.fi.muni.cz/trac/noske)\n- [NoSketch Engine Docker](https://github.com/ELTE-DH/NoSketch-Engine-Docker)\n- [Sketch Grammar Explorer](https://github.com/engisalor/sketch-grammar-explorer)\n- [Plotly graphing library](https://plotly.com/python/)\n- [Dash web app framework](https://dash.plotly.com/)\n- [Docker containerization](https://www.docker.com/)\n\n## Getting started\n\n1. Clone the repo\n2. Set up environment variables in `.env` (copy and rename `.env-example` to get started)\n3. Set up configuration files in `config/` and make a `data/` directory for storing data\n4. Option 1: run Quartz directly as a Flask app (for testing or local usage without Docker, e.g., `set -a \u0026\u0026 source .env \u0026\u0026 set +a \u0026\u0026 python app.py`); visit the app at `http://127.0.0.1:8080/` and make a test query\n5. Option 2: build and use the Docker image `docker-compose up` (doesn't have access to localhost by default)\n\nTo make queries to the Sketch Engine server, get an [API key](https://www.sketchengine.eu/documentation/api-documentation/) and review their [fair use policy](https://www.sketchengine.eu/fair-use-policy/).\n\nTo work with your own server, check out NoSketch Engine. Accessing any corpus on any (No)SkE server should work as long as the config file is properly defined.\n\n\u003eWarning: Quartz expects the (No)SkE server to be available when the app/container is first started and fails if otherwise. On startup it makes initial API calls to collect corpus information. Once those calls are cached, having server access isn't technically required to view cached queries.\n\n### Environment variables\n\nThis example uses the Susanne corpus on Sketch Engine.\n\nQuartz expects a few environment variables to be available. Set these up by renaming `.env-example` to `.env` and adapt as needed.\n\nSome important environment variables:\n\n1. A YAML configuration file is needed to define which corpora are available\n    - `CORPORA_YML=config/corpora-ske.yml`\n2. A server to interact with\n    - `SGEX_SERVER=ske` points to Sketch Engine's server\n    - or use a full URL to a server`SGEX_SERVER=https://api.sketchengine.eu/bonito/run.cgi`\n3. A username and API key for the server, if required\n    - `SGEX_API_KEY=\"\u003cKEY\u003e\"`\n    - `SGEX_USERNAME=\"\u003cUSER\u003e\"`\n4. Bind paths for configuration and data files\n    - `CONFIG_DIR=/current/working/directory/config`\n    - `DATA_DIR=/current/working/directory/data`\n\n### Corpora configuration file\n\nA YAML file is needed with details about each corpus. This example includes the SkE Susanne corpus - create one or more config files to define sets of corpora to study together.\n\n```yaml\n# settings for corpora\n# corpus name used by API\npreloaded/susanne:\n  # name shown in graphs\n  name: Susanne\n  # unique color code\n  color: \"#636EFA\"\n  # corpus description file (optional)\n  md_file: config/susanne.md\n  # text types to exclude\n  exclude:\n    - doc.wordcount\n    - font.type\n  # text type labels (required for every non-excluded text type/attribute)\n  label:\n    doc.file: file\n    doc.n: \"n\"\n    head.type: head\n  # text types considered comparable with other corpora\n  # comparable:\n    # - \u003cVALUE\u003e\n  # text types to visualize w/ choropleth (requires ISO3 strings, case insensitive)\n  # choropleth:\n    # - \u003cVALUE\u003e\n```\n\n### Trying out the app\n\nOnce `.env` and the YAML configuration file are ready, start the app and make a query, for example `with; without`. Two bar graphs will appear showing frequencies in the Susanne corpus in several text types. The app currently has a page for making queries, one for inspecting corpora, and a user guide.\n\n![image](/quartz-app.png)\n\nQuartz can be used without much knowledge of corpus linguistics, but a range of complex corpus queries are also possible. That said, interpreting results is a question left to the researcher: Quartz just visualizes frequency data. It's a good idea to check its results directly in Sketch Engine to ensure frequencies are computed properly (so far it's been reliable for hundreds of queries).\n\n### Development\n\nAPI-based data collection requires understanding the [Sketch Grammar Explorer](https://github.com/engisalor/sketch-grammar-explorer) package (SGEX, a Sketch Engine API wrapper); try it out as a standalone tool if you plan on doing custom data processing. Also see [/components/freqs_fig.py](/components/freqs_fig.py) for examples of how to write custom visualizations with Plotly and SkE API data.\n\n## About\n\nQuartz was developed as part of work at the [Humanitarian Encyclopedia](https://humanitarianencyclopedia.org) in coordination with the University of Granada [LexiCon research group](http://lexicon.ugr.es). It's the upstream repository for the [Humanitarian Encyclopedia Dashboard](https://humanitarianencyclopedia.org/analysis) ([source code](https://github.com/Humanitarian-Encyclopedia/he-dashboard)). If you're interested in the Dashboard or studying humanitarian discourse, make a free account at the Encyclopedia to try it out.\n\nQuartz relies on APIs made available thanks to the work of [Lexical Computing](https://www.lexicalcomputing.com/) and [Sketch Engine contributors](https://www.sketchengine.eu/bibliography-of-sketch-engine/). The [Docker image](https://github.com/ELTE-DH/NoSketch-Engine-Docker) from Eötvös Loránd University Department of Digital Humanities is also quite helpful.\n\nThis app includes [Dash Bootstrap Components](https://dash-bootstrap-components.opensource.faculty.ai/); also check out [Dash's community forum](https://community.plotly.com/) for tips on visualization techniques.\n\nThe name Quartz is a nod to Sketch Engine's GUI, Crystal.\n\n## Citation\n\nSee [this article](https://doi.org/10.3390/languages9030081):\n\n```bibtex\n@article{isaacsQuartzTemplateQuantitative2024,\n\ttitle = {Quartz: {A} template for quantitative corpus data visualization tools},\n\tvolume = {9},\n\tdoi = {10.3390/languages9030081},\n\tnumber = {(3):81},\n\tjournal = {Languages},\n\tauthor = {Isaacs, Loryn and Odlum, Alex and León-Araúz, Pilar},\n}\n```\n\nTo cite Quartz as software, see [CITATION.cff](/CITATION.cff).\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fengisalor%2Fquartz","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fengisalor%2Fquartz","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fengisalor%2Fquartz/lists"}