{"id":23370032,"url":"https://github.com/maastrichtu-ids/dsri-documentation","last_synced_at":"2025-06-15T07:32:51.680Z","repository":{"id":37853502,"uuid":"211358427","full_name":"MaastrichtU-IDS/dsri-documentation","owner":"MaastrichtU-IDS","description":"📖 Documentation for the Data Science Research Infrastructure at Maastricht University","archived":false,"fork":false,"pushed_at":"2025-04-01T09:53:24.000Z","size":223913,"stargazers_count":21,"open_issues_count":11,"forks_count":9,"subscribers_count":11,"default_branch":"master","last_synced_at":"2025-04-06T19:59:21.689Z","etag":null,"topics":["data-science","data-science-research","documentation","dsri","kubernetes","openshift"],"latest_commit_sha":null,"homepage":"https://dsri.maastrichtuniversity.nl","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MaastrichtU-IDS.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-09-27T16:18:10.000Z","updated_at":"2025-04-01T09:51:05.000Z","dependencies_parsed_at":"2023-02-18T16:45:41.459Z","dependency_job_id":"2e777bf8-e679-49aa-b7ac-a4da840201ed","html_url":"https://github.com/MaastrichtU-IDS/dsri-documentation","commit_stats":{"total_commits":882,"total_committers":13,"mean_commits":67.84615384615384,"dds":"0.17913832199546487","last_synced_commit":"d0ee4945707e5541113e3230e35492a147174df7"},"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MaastrichtU-IDS%2Fdsri-documentation","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MaastrichtU-IDS%2Fdsri-documentation/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MaastrichtU-IDS%2Fdsri-documentation/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MaastrichtU-IDS%2Fdsri-documentation/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MaastrichtU-IDS","download_url":"https://codeload.github.com/MaastrichtU-IDS/dsri-documentation/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248251310,"owners_count":21072688,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-science","data-science-research","documentation","dsri","kubernetes","openshift"],"created_at":"2024-12-21T15:32:54.047Z","updated_at":"2025-06-15T07:32:51.669Z","avatar_url":"https://github.com/MaastrichtU-IDS.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Build](https://github.com/MaastrichtU-IDS/dsri-documentation/workflows/Publish%20to%20GitHub%20Pages/badge.svg)](https://github.com/MaastrichtU-IDS/dsri-documentation/actions?query=workflow%3A%22Publish+to+GitHub+Pages%22) [![Slack](https://img.shields.io/badge/Chat%20on-Slack-blueviolet)](https://dsri.slack.com)\n\nThe documentation website at [dsri.maastrichtuniversity.nl](https://dsri.maastrichtuniversity.nl/) is automatically updated by a [GitHub Action](/actions) at each push to this `master` branch.\n\n## 📝 Contribute\n\nContributions are welcome! See the [guidelines to contribute to the website](https://dsri.maastrichtuniversity.nl/docs/contribute).\n\n### Edit documentation pages\n\nEditing a documentation file is as easy as going to https://github.com/MaastrichtU-IDS/dsri-documentation/edit/master/website/docs/introduction.md.\n\n* Edit a page by login with an account that has edit permissions.\n\n* Otherwise fork the repository and modify the files you want. Pull requests are welcome!\n\nBrowse all documentation pages [here](https://github.com/MaastrichtU-IDS/dsri-documentation/tree/master/website/docs).\n\n\u003e We recommend using [Typora](https://typora.io/) to edit [markdown](https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet) files on your computer.\n\n### Files locations\n\n- All documentation pages are in `website/docs`\n- Add new `docs` pages links to the sidebar: [website/sidebars.json](https://github.com/MaastrichtU-IDS/d2s-docs/blob/master/website/sidebars.json).\n- Main parameters of the website can be found in [website/docusaurus.config.js](https://github.com/MaastrichtU-IDS/d2s-docs/blob/master/website/docusaurus.config.js).\n- Static content (any resource to download, images, css, js) can be provided in [website/static](https://github.com/MaastrichtU-IDS/d2s-docs/tree/master/website/static)\n- Pages other than `docs` are in [website/src/pages](https://github.com/MaastrichtU-IDS/d2s-docs/tree/master/website/src/pages) (e.g. `help.md` or `index.tsx`)\n\n### Add an announcement\n\nYou can easily add a general announcement bar on the website if you want to pass some information to your users, like dates of maintenance\n\nOpen the file `website/docusaurus.config.js` and update the `announcementBar` variable.\n\nYou can comment the `announcementBar` code block when you want to remove the announcement bar.\n\n## 🧑‍💻 Run for development\n\n### Just run the website\n\nTo check changes in the documentation, go to the `/website` directory and start the website on http://localhost:19006 using the production API for user statistics:\n\n```shell\ncd website\n\nyarn install\n\nyarn start\n```\n\n### Run the full stack\n\nTo run the full stack including the database and API, we use docker-compose\n\n1. Define the `.env` file to change the default configuration (user credential to enable/disable GPU on the cluster, Slack config):\n\n   ``` bash\n   # Virtual hosts\n   ADMIN_VIRTUAL_HOST=admin.dsri.test.unimaas.nl\n   API_VIRTUAL_HOST=api.dsri.test.unimaas.nl\n   CALENDAR_VIRTUAL_HOST=calendar.dsri.test.unimaas.nl\n   BOOKING_VIRTUAL_HOST=booking.dsri.test.unimaas.nl\n   DSRI_VIRTUAL_HOST=dsri.test.unimaas.nl\n\n   # Lets encrypt hosts\n   ADMIN_LETSENCRYPT_HOST=admin.dsri.test.unimaas.nl\n   API_LETSENCRYPT_HOST=api.dsri.test.unimaas.nl\n   CALENDAR_LETSENCRYPT_HOST=calendar.dsri.test.unimaas.nl\n   BOOKING_LETSENCRYPT_HOST=booking.dsri.test.unimaas.nl\n   DSRI_LETSENCRYPT_HOST=dsri.test.unimaas.nl\n\n   # Lets encrypt test\n   LETSENCRYPT_TEST=true\n\n   # MySQL DB\n   DB_NAME=dsri-db\n   DB_USER=dsri-user\n   DB_PASSWORD=password\n   DB_HOST=mysql\n\n   # API\n   API_PASSWORD=password\n   CLUSTER_URL=\"\"\n   CLUSTER_API_KEY=\"\"\n\n   # Slack\n   SLACK_BOT_TOKEN=aaaa-0000000000-0000000000-0000000000\n   SLACK_CHANNEL=CXXXXXXXXXXX      \n\n   # SMTP\n   SMTP_FROM=email@email.nl\n   SMTP_HOST=\n   SMTP_PORT=\n   ```\n\n###\n\n2. Run the stack:\n\n   * API on http://localhost:8000, automatically reloaded on change to the code, with CRON job enabled\n   * The GPU calendar on http://localhost:8001\n   * Database accessible through phpMyAdmin on http://localhost:8002\n\n    ```bash\n   # Don't forget to switch to the right volumes for development!\n\n   # Build images for dsri-api, cron, gpu-calender and gpu-reservation\n   docker compose build --no-cache\n\n   # Start the containers (detached), you can also choose to up specific services e.g. append: nginx_proxy dsri-api \n   docker compose up -d --force-recreate\n   ```\n\n   \u003e ⚠️ The first time you start the stack you might need to stop and restart the stack once the SQL database has been initialized for the API to properly connect to the database\n\n###\n\n3. In another terminal, run the website on http://localhost:3000, it will use the local API to display stats:\n\n   ```bash\n   cd website\n\n   yarn install\n\n   yarn dev\n   ```\n\n### Update dependencies\n\nDependabot will automatically create pull requests to update libraries containing a known vulnerability that have been fixed in newer version.\n\nYou can also use `yarn` to automatically upgrade packages that can:\n\n```bash\nyarn upgrade\n```\n\nAlternatively you can also change the packages versions requirements in the `package.json` and run `yarn`\n\n###\n\n## 🚀 Deploy in production\n\n### Deploy the frontend to GitHub pages\n\nThe documentation website at [dsri.maastrichtuniversity.nl](https://dsri.maastrichtuniversity.nl/) is automatically updated by a [GitHub Action](https://github.com/MaastrichtU-IDS/dsri-documentation/blob/master/actions) at each push to the `master` branch of this repository.\n\nThe GitHub Action will automatically compile the website to HTML in the `gh-page` branch which is served by GitHub Page.\n\n### Deploy the backend on a server\n\nDefine the `.env` file to change the default configuration (user credential to enable/disable GPU on the cluster, Slack config):\n\n``` bash\n# Virtual hosts\nADMIN_VIRTUAL_HOST=admin.dsri.test.unimaas.nl\nAPI_VIRTUAL_HOST=api.dsri.test.unimaas.nl\nCALENDAR_VIRTUAL_HOST=calendar.dsri.test.unimaas.nl\nBOOKING_VIRTUAL_HOST=booking.dsri.test.unimaas.nl\nDSRI_VIRTUAL_HOST=dsri.test.unimaas.nl\n\n# Lets encrypt hosts\nADMIN_LETSENCRYPT_HOST=admin.dsri.test.unimaas.nl\nAPI_LETSENCRYPT_HOST=api.dsri.test.unimaas.nl\nCALENDAR_LETSENCRYPT_HOST=calendar.dsri.test.unimaas.nl\nBOOKING_LETSENCRYPT_HOST=booking.dsri.test.unimaas.nl\nDSRI_LETSENCRYPT_HOST=dsri.test.unimaas.nl\n\n# Lets encrypt test\nLETSENCRYPT_TEST=true\n\n# MySQL DB\nDB_NAME=dsri-db\nDB_USER=dsri-user\nDB_PASSWORD=password\nDB_HOST=mysql\n\n# API\nAPI_PASSWORD=password\nCLUSTER_URL=\"\"\nCLUSTER_API_KEY=\"\"\n\n# Slack\nSLACK_BOT_TOKEN=aaaa-0000000000-0000000000-0000000000\nSLACK_CHANNEL=CXXXXXXXXXXX      \n\n# SMTP\nSMTP_FROM=email@email.nl\nSMTP_HOST=\nSMTP_PORT=\n```\n\nStart the docker-compose in production using [nginx-proxy](https://github.com/nginx-proxy/nginx-proxy).\n\n```bash\n# Build images for dsri-api, cron, gpu-calender and gpu-reservation\ndocker compose build --no-cache\n\n# Start the containers (detached), you can also choose to up specific services e.g. append: nginx_proxy dsri-api \ndocker compose up -d --force-recreate\n```\n\n###\n\n### Database setup\n\n#### Import CSV\n\nTo import a CSV of users in the database: remove the header, set `created_at` and `use_dsri_date`as a `VARCHAR(255)`, import the CSV file via phpMyAdmin, then set back `created_at` and `use_dsri_date` as `DATETIME`\n\n```sql\nUPDATE user SET created_at = STR_TO_DATE(created_at, '%d-%m-%Y %H:%i:%s');\nUPDATE user SET use_dsri_date = STR_TO_DATE(use_dsri_date, '%d-%m-%Y');\n```\n\n#### Create new database users\n\nLogin with the `root` user, and click on the **SQL** tab\n\nReplace `username` by the username, and `password` by the password (thanks captain obvious):\n\nCommands to create a read-only user:\n\n```sql\nDROP USER IF EXISTS 'username'@'%';\nCREATE USER 'username'@'%' IDENTIFIED BY 'password';\nGRANT USAGE ON *.* TO 'username'@'%' IDENTIFIED BY 'password';\nGRANT SELECT ON `dsri-db`.* TO 'username'@'%';\nFLUSH PRIVILEGES;\n```\n\nOr give full access:\n\n```sql\nGRANT ALL PRIVILEGES ON `dsri-db`.* TO 'username'@'%';\n```\n\n### Change your user password\n\nTo change the password of your database user, click on the **SQL** tab, and execute:\n\n```sql\nSET PASSWORD FOR 'username'@'%' = PASSWORD('newpassword');\n```\n\n###\n\n### Backup\n\n#### Export as CSV\n\n1. Go to phpMyAdmin \u003e click on the `dsri-db` database \u003e go to the **Export** tab\n2. Change the **Export method** to **Custom**\n3. Change the **Format** to **CSV** or **CSV for MS Excel**\n4. Check **Export tables as separate files**\n4. Click **Go**\n\n#### Complete database backup\n\nA CSV backup of the database is generated every week by a CRON job (cf. https://github.com/MaastrichtU-IDS/dsri-documentation/blob/master/server/api/main.py#L52) and stored locally on the server where the service is deployed.\n\nFor reliability reason we choose to export as CSV: open, lightweight, hard to corrupt, and contains all information we need to backup (otherwise we would need to fight with weird backup tools poorly built by lost sysadmins that fails most of the time just to save some useless indexes, which just leads to increased chances of corrupting the data we backup...)\n\n## 💡 Markdown tips\n\n```markdown\n:::note\nGrey box\n:::\n:::tip You can specify an optional title\nGreen box\n:::\n:::info\nBlue box\n:::\n:::caution\nOrange bpx\n:::\n:::danger\nRed box\n:::\n```\n\nEmbed a Google docs presentation (size does not change dynamically). You can check the [\"documentation\" by gitlab](https://about.gitlab.com/handbook/markdown-guide/#google-slides) but it does not work (how can they expect to get a responsive website by providing hardcoded pixel size?):\n\n```html\n\u003cfigure class=\"video_container\"\u003e\n  \u003ciframe src=\"https://docs.google.com/presentation/d/e/2PACX-1vRAfZdOfGt761tIAj2e35OYrOL4uIKWiAQB15MXvsqso3XJ5Mr3-W4dOa9KjDTZpi1LE_D2CU1F5Thy/embed?start=false\u0026loop=false\u0026delayms=15000\" frameborder=\"0\" width=\"960\" height=\"569\" allowfullscreen=\"true\" mozallowfullscreen=\"true\" webkitallowfullscreen=\"true\"\u003e\u003c/iframe\u003e\n\u003c/figure\u003e\n\n# With CSS:\n\u003cscript\u003e\n.video-container{\n  position: absolute;\n  top: 0%;\n  left: 0%;\n  height: 100%;\n  width: 100%;\n  /* overflow: hidden; */\n}\n\u003c/script\u003e\n```\n\nConvert a mkv video to webm (better for direct embedding):\n\n```bash\ncd website/static/img/\n\nffmpeg -i video_dsri_introduction.mkv -c:v libvpx -crf 10 -c:a libvorbis video_dsri_introduction.webm\n```\n\n## 🧜‍♀️ Mermaid sourcecodes\n\nRequest access to UM internal servers\n\n```\nsequenceDiagram\n    Researcher-\u003e\u003e+DSRI-team: Request DSRI access to UM system\n    DSRI-team-\u003e\u003e+UM-SOC: Request DSRI access to UM system\n    UM-SOC--\u003e\u003e+DSRI-team: Go ahead\n    DSRI-team--\u003e\u003e+System admin:\n\n    sequenceDiagram\n    Researcher-\u003e\u003e+DSRI-team: Request DSRI access to UM system\n    DSRI-team-\u003e\u003e+UM-SOC: Request DSRI access to UM system\n    UM-SOC--\u003e\u003e+DSRI-team: Go ahead\n    DSRI-team-\u003e\u003e+UM System admin: Request access from DSRI\n    UM System admin--\u003e\u003e+ICTS: Request change in firewall\n    DSRI-team--\u003e\u003e+ICTS: Request change in firewall\n    DSRI-team-\u003e\u003e+Researcher: UM system accessible from DSRI\n```\n\n## 🙏 Acknowledgments\n\nDocumentation website generated using [Docusaurus](https://docusaurus.io/).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmaastrichtu-ids%2Fdsri-documentation","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmaastrichtu-ids%2Fdsri-documentation","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmaastrichtu-ids%2Fdsri-documentation/lists"}