{"id":24511751,"url":"https://github.com/scribe-org/scribe-server","last_synced_at":"2025-10-30T10:32:21.575Z","repository":{"id":183802049,"uuid":"659612272","full_name":"scribe-org/Scribe-Server","owner":"scribe-org","description":"Backend service for Scribe data downloads","archived":false,"fork":false,"pushed_at":"2025-09-08T05:44:01.000Z","size":545,"stargazers_count":9,"open_issues_count":5,"forks_count":8,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-09-08T07:21:17.212Z","etag":null,"topics":["api","autosuggest","backend","data","data-downloader","data-pipeline","dictionary","education","elt","emoji","go","golang","grammar","language","learning","open-source","translation","wikidata","wikipedia"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/scribe-org.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.txt","code_of_conduct":".github/CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2023-06-28T07:43:42.000Z","updated_at":"2025-09-08T05:44:05.000Z","dependencies_parsed_at":null,"dependency_job_id":"6d51d3a7-d227-4dcd-9bea-b63e59b6f9dd","html_url":"https://github.com/scribe-org/Scribe-Server","commit_stats":null,"previous_names":["scribe-org/scribe-server"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/scribe-org/Scribe-Server","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scribe-org%2FScribe-Server","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scribe-org%2FScribe-Server/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scribe-org%2FScribe-Server/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scribe-org%2FScribe-Server/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/scribe-org","download_url":"https://codeload.github.com/scribe-org/Scribe-Server/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scribe-org%2FScribe-Server/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":274887046,"owners_count":25368228,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-12T02:00:09.324Z","response_time":60,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api","autosuggest","backend","data","data-downloader","data-pipeline","dictionary","education","elt","emoji","go","golang","grammar","language","learning","open-source","translation","wikidata","wikipedia"],"created_at":"2025-01-22T00:41:21.999Z","updated_at":"2025-10-30T10:32:21.554Z","avatar_url":"https://github.com/scribe-org.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n  \u003ca href=\"https://github.com/scribe-org/Scribe-Server\"\u003e\u003cimg src=\"https://raw.githubusercontent.com/scribe-org/Scribe-Server/main/.github/resources/images/ScribeServerGitHubOrgBanner.png\" width=1024 alt=\"Scribe-Server Logo\"\u003e\u003c/a\u003e\n\u003c/div\u003e\n\n[![platforms](https://img.shields.io/static/v1?message=Toolforge\u0026logo=wikimedia-foundation\u0026color=990000\u0026logoColor=white\u0026label=%20)](https://github.com/scribe-org/Scribe-Server)\n[![issues](https://img.shields.io/github/issues/scribe-org/Scribe-Server?label=%20\u0026logo=github)](https://github.com/scribe-org/Scribe-Server/issues)\n[![language](https://img.shields.io/badge/Go%201.20-00ADD8.svg?logo=go\u0026logoColor=ffffff)](https://github.com/scribe-org/Scribe-Server/blob/main/CONTRIBUTING.md)\n[![license](https://img.shields.io/github/license/scribe-org/Scribe-Server.svg?label=%20)](https://github.com/scribe-org/Scribe-Server/blob/main/LICENSE.txt)\n[![coc](https://img.shields.io/badge/Contributor%20Covenant-ff69b4.svg)](https://github.com/scribe-org/Scribe-Server/blob/main/.github/CODE_OF_CONDUCT.md)\n[![mastodon](https://img.shields.io/badge/Mastodon-6364FF.svg?logo=mastodon\u0026logoColor=ffffff)](https://wikis.world/@scribe)\n[![matrix](https://img.shields.io/badge/Matrix-000000.svg?logo=matrix\u0026logoColor=ffffff)](https://matrix.to/#/#scribe_community:matrix.org)\n\n### Backend service for Scribe data downloads\n\n**Scribe-Server** is a backend service that provides the API by which data is available for download within Scribe apps. The goal is to create a [Scribe-Data](https://github.com/scribe-org/Scribe-Data) based regularly updating dataset that can signal new data availability as well as allow for language pack downloads. Scribe-Server can be accessed via [scribe-server.toolforge.org](https://scribe-server.toolforge.org/), with the SQLite data packs being available for download via [scribe-server.toolforge.org/packs/sqlite](https://scribe-server.toolforge.org/packs/sqlite/).\n\n\u003e [!NOTE]\\\n\u003e The [contributing](#contributing) section has information for those interested, with the articles and presentations in [featured by](#featured-by) also being good resources for learning more about Scribe.\n\nScribe apps are available on [iOS](https://github.com/scribe-org/Scribe-iOS), [Android](https://github.com/scribe-org/Scribe-Android) (planned) and [Desktop](https://github.com/scribe-org/Scribe-Desktop) (planned). For the data formatting processes see [Scribe-Data](https://github.com/scribe-org/Scribe-Data).\n\nCheck out Scribe's [architecture diagrams](https://github.com/scribe-org/Organization/blob/main/ARCHITECTURE.md) for an overview of the organization including our applications, services and processes. It depicts the projects that [Scribe](https://github.com/scribe-org) is developing as well as the relationships between them and the external systems with which they interact.\n\n\u003ca id=\"contents\"\u003e\u003c/a\u003e\n\n# **Contents**\n\n- [Contributing](#contributing)\n- [Environment Setup](#environment-setup)\n- [Supported Languages](#supported-languages)\n- [Featured By](#featured-by)\n\n\u003ca id=\"contributing\"\u003e\u003c/a\u003e\n\n# Contributing [`⇧`](#contents)\n\n\u003ca href=\"https://matrix.to/#/#scribe_community:matrix.org\"\u003e\n  \u003cimg src=\"https://raw.githubusercontent.com/scribe-org/Organization/main/resources/images/logos/MatrixLogoGrey.png\" width=\"175\" alt=\"Public Matrix Chat\" align=\"right\"\u003e\n\u003c/a\u003e\n\nScribe uses [Matrix](https://matrix.org/) for communications. You're more than welcome to [join us in our public chat rooms](https://matrix.to/#/#scribe_community:matrix.org) to share ideas, ask questions or just say hi to the team :) We'd suggest that you use the [Element](https://element.io/) client and [Element X](https://element.io/app) for a mobile app.\n\nPlease see the [contribution guidelines](https://github.com/scribe-org/Scribe-Server/blob/main/CONTRIBUTING.md) if you are interested in contributing to Scribe-Server. Work that is in progress or could be implemented is tracked in the [issues](https://github.com/scribe-org/Scribe-Server/issues) and [projects](https://github.com/scribe-org/Scribe-Server/projects).\n\n\u003e [!NOTE]\\\n\u003e Just because an issue is assigned on GitHub doesn't mean the team isn't open to your contribution! Feel free to write [in the issues](https://github.com/scribe-org/Scribe-Server/issues) and we can potentially reassign it to you.\n\nThose interested can further check the [`-next release-`](https://github.com/scribe-org/Scribe-Server/labels/-next%20release-) and [`-priority-`](https://github.com/scribe-org/Scribe-Server/labels/-priority-) labels in the [issues](https://github.com/scribe-org/Scribe-Server/issues) for those that are most important, as well as those marked [`good first issue`](https://github.com/scribe-org/Scribe-Server/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22) that are tailored for first-time contributors. For those new to coding or our tech stack, we've collected [links to helpful documentation pages](https://github.com/scribe-org/Scribe-Server/blob/main/CONTRIBUTING.md#learning-the-tech) in the [contribution guidelines](https://github.com/scribe-org/Scribe-Server/blob/main/CONTRIBUTING.md).\n\nAfter your first few pull requests organization members would be happy to discuss granting you further rights as a contributor, with a maintainer role then being possible after continued interest in the project. Scribe seeks to be an inclusive and supportive organization. We'd love to have you on the team!\n\n### Ways to Help [`⇧`](#contents)\n\n- [Reporting bugs](https://github.com/scribe-org/Scribe-Server/issues/new?assignees=\u0026labels=bug\u0026template=bug_report.yml) as they're found 🐞\n- Working on [new features](https://github.com/scribe-org/Scribe-Server/issues?q=is%3Aissue+is%3Aopen+label%3Afeature) ✨\n- [Localization](https://github.com/scribe-org/Scribe-Server/issues?q=is%3Aissue+is%3Aopen+label%3Alocalization) for the app and Google Play 🌐\n- [Documentation](https://github.com/scribe-org/Scribe-Server/issues?q=is%3Aissue+is%3Aopen+label%3Adocumentation) for onboarding and project cohesion 📝\n- Adding language data to [Scribe-Data](https://github.com/scribe-org/Scribe-Data/issues) via [Wikidata](https://www.wikidata.org/)! 🗃️\n\n### Road Map [`⇧`](#contents)\n\nThe Scribe road map can be followed in the organization's [project board](https://github.com/orgs/scribe-org/projects/1) where we list the most important issues along with their priority, status and an indication of which sub projects they're included in (if applicable).\n\n\u003e [!NOTE]\\\n\u003e Consider joining our [bi-weekly developer syncs](https://etherpad.wikimedia.org/p/scribe-dev-sync)!\n\n### Data Edits [`⇧`](#contents)\n\n\u003e [!NOTE]\\\n\u003e Please see the [Wikidata and Scribe Guide](https://github.com/scribe-org/Organization/blob/main/WIKIDATAGUIDE.md) for an overview of [Wikidata](https://www.wikidata.org/) and how Scribe uses it.\n\nScribe does not accept direct edits to the grammar JSON files as they are sourced from [Wikidata](https://www.wikidata.org/). Edits can be discussed and the [Scribe-Data](https://github.com/scribe-org/Scribe-Data) queries will be changed and ran before an update. If there is a problem with one of the files, then the fix should be made on [Wikidata](https://www.wikidata.org/) and not on Scribe. Feel free to let us know that edits have been made by [opening a data issue](https://github.com/scribe-org/Scribe-Server/issues/new?assignees=\u0026labels=data\u0026template=data_wikidata.yml) or contacting us in the [issues for Scribe-Data](https://github.com/scribe-org/Scribe-Data/issues) and we'll be happy to integrate them!\n\n\u003ca id=\"environment-setup\"\u003e\u003c/a\u003e\n\n# Environment Setup [`⇧`](#contents)\n\nScribe-Server is developed using the [Go](https://go.dev/) programming language. Those new to Go or wanting to develop their skills are more than welcome to contribute! The first step on your Go journey would be to read through the [Go documentation](https://go.dev/doc), with the [Effective Go](https://go.dev/doc/effective_go) page in particular having great insights into the language's good practices and standards. The general steps to setting up a development environment are:\n\n1. Download and install [Go](https://go.dev/doc/install)\n\n2. [Fork](https://docs.github.com/en/get-started/quickstart/fork-a-repo) the [Scribe-Server repo](https://github.com/scribe-org/Scribe-Server), clone your fork, and configure the remotes:\n\n\u003e [!NOTE]\n\u003e\n\u003e \u003cdetails\u003e\u003csummary\u003eConsider using SSH\u003c/summary\u003e\n\u003e\n\u003e \u003cp\u003e\n\u003e\n\u003e Alternatively to using HTTPS as in the instructions below, consider SSH to interact with GitHub from the terminal. SSH allows you to connect without a user-pass authentication flow.\n\u003e\n\u003e To run git commands with SSH, remember then to substitute the HTTPS URL, `https://github.com/...`, with the SSH one, `git@github.com:...`.\n\u003e\n\u003e - e.g. Cloning now becomes `git clone git@github.com:\u003cyour-username\u003e/Scribe-Server.git`\n\u003e\n\u003e GitHub also has their documentation on how to [Generate a new SSH key](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent) 🔑\n\u003e\n\u003e \u003c/p\u003e\n\u003e \u003c/details\u003e\n\n```bash\n# Clone your fork of the repo into the current directory.\ngit clone https://github.com/\u003cyour-username\u003e/Scribe-Server.git\n# Navigate to the newly cloned directory.\ncd Scribe-Server\n# Assign the original repo to a remote called \"upstream\".\ngit remote add upstream https://github.com/scribe-org/Scribe-Server.git\n```\n\n- Now, if you run `git remote -v` you should see two remote repositories named:\n  - `origin` (forked repository)\n  - `upstream` (Scribe-Server repository)\n\n3. Navigate to the root directory of the project\n\n4. Create a `config.yaml` file with the configuration needed for the project\n\n   - Reference the [`config-example.yaml`](./config-example.yaml) to get started\n\n## Important Note on PATH Configuration for Go Tools\n\n\u003e After installing `Go`, it's highly recommended to add your `Go` binary directory (`$(go env GOPATH)/bin`) to your system's `PATH`. This ensures that tools installed via `go install` (like `swag`, `oapi-codegen`, etc.) are directly accessible from your terminal.\n\nYou can typically do this by adding the following line to your shell configuration file (e.g., `~/.bashrc`, `~/.zshrc`, or `~/.profile`):\n\n```bash\nexport PATH=$(go env GOPATH)/bin:$PATH\n```\n\nAfter adding this line, remember to apply the changes by sourcing the file (e.g., `source ~/.bashrc`) or by opening a new terminal session.\n\n5. Install [MariaDB](https://mariadb.com/) locally via its [installation guide](https://mariadb.com/docs/server/server-management/install-and-upgrade-mariadb/installing-mariadb/binary-packages).\n\n   - Create a database using the `database.name` value from your `config.yaml` with the following commands (using Homebrew, for example):\n\n   ```bash\n   brew services start mariadb\n   mariadb -u root  # you may need to sudo this command\n\n   # To stop the server:\n   brew services stop mariadb\n   ```\n\n   - You can now run the commands found in [CREATE_SCRIBE_SERVER_DB.md](./CREATE_SCRIBE_SERVER_DB.md) to make the needed MariaDB database.\n\n6. Start a local Scribe-Server:\n\n   ```bash\n   # Run the following target from the 'Makefile'.\n   # Migrate SQLite files from Scribe-Data to MariaDB for use in Scribe-Server:\n   make build\n   make migrate\n   # Start Scribe-Server on your local host:\n   make run\n   ```\n\n   - NOTE: This `make` target simply runs `go run .` on the project\n   - Scribe-Server should now be running locally!\n\n7. To generate the documentation for Scribe-Server, please run the following:\n\n   ```bash\n   make docs\n   ```\n\n   Once the server is running (via `make run` or `make dev`), you can access the API documentation at:\n\n   - Swagger UI: http://localhost:8080/swagger/index.html\u003c/br\u003e\n   - Alternative docs: http://localhost:8080/docs/index.html\n\n\u003e [!NOTE]\n\u003e Feel free to contact the team in the [Data room on Matrix](https://matrix.to/#/#ScribeData:matrix.org) if you're having problems getting your environment setup!\n\n\u003ca id=\"supported-languages\"\u003e\u003c/a\u003e\n\n# Supported Languages [`⇧`](#contents)\n\nScribe's goal is functional, feature-rich keyboards for all languages. You can check the currently available languages and data for Scribe applications on our website at [scri.be/docs/server/available-data](https://scri.be/docs/server/available-data).\n\nSee [scribe_data/wikidata/language_data_extraction](https://github.com/scribe-org/Scribe-Data/tree/main/src/scribe_data/wikidata/language_data_extraction) for queries in the [Scribe-Data](https://github.com/scribe-org/Scribe-Data) project for currently supported languages and those that have substantial data on [Wikidata](https://www.wikidata.org/). Also see the [`new keyboard`](https://github.com/scribe-org/Scribe-iOS/issues?q=is%3Aissue+is%3Aopen+label%3A%22new+keyboard%22) label in the [Issues](https://github.com/scribe-org/Scribe-iOS/issues) for keyboards that are currently in progress or being discussed, and [suggest a new keyboard](https://github.com/scribe-org/Scribe-iOS/issues/new?assignees=\u0026labels=new+keyboard\u0026template=new_keyboard.yml\u0026title=Add+%3Clanguage%3E+keyboard) if you don't see it being worked on already!\n\n\u003ca id=\"featured-by\"\u003e\u003c/a\u003e\n\n# Featured By [`⇧`](#contents)\n\nPlease see the [blog posts page on our website](https://scri.be/docs/about/blog-posts) for a list of articles on Scribe, and feel free to open a pull request to add one that you've written at [scribe-org/scri.be](github.com/scribe-org/scri.be)!\n\n### Organizations\n\nThe following organizations have supported the development of Scribe projects through various programs. Thank you all! 💙\n\n\u003cdiv align=\"center\"\u003e\n  \u003cbr\u003e\n    \u003ca href=\"https://tech-news.wikimedia.de/en/2022/03/18/lexicographical-data-for-language-learners-the-wikidata-based-app-scribe/\"\u003e\u003cimg width=\"180\" src=\"https://raw.githubusercontent.com/scribe-org/Organization/main/resources/images/logos/WikimediaDeutschlandLogo.png\" alt=\"Wikimedia Deutschland logo linking to an article on Scribe in the tech news blog.\"\u003e\u003c/a\u003e\n    \u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\n    \u003ca href=\"https://www.mediawiki.org/wiki/New_Developers#Scribe\"\u003e\u003cimg width=\"180\" src=\"https://raw.githubusercontent.com/scribe-org/Organization/main/resources/images/logos/WikimediaFoundationLogo.png\" alt=\"Wikimedia Foundation logo linking to the MediaWiki new developers page.\"\u003e\u003c/a\u003e\n    \u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\n  \u003cbr\u003e\n\u003c/div\u003e\n\n\u003cdiv align=\"center\"\u003e\n  \u003cbr\u003e\n    \u003ca href=\"https://summerofcode.withgoogle.com/\"\u003e\u003cimg width=\"140\" src=\"https://raw.githubusercontent.com/scribe-org/Organization/main/resources/images/logos/GSoCLogo.png\" alt=\"Google Summer of Code logo linking to its website.\"\u003e\u003c/a\u003e\n    \u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\n    \u003ca href=\"https://www.outreachy.org/\"\u003e\u003cimg width=\"350\" src=\"https://raw.githubusercontent.com/scribe-org/Organization/main/resources/images/logos/OutreachyLogo.png\" alt=\"Outreachy logo linking to its website.\"\u003e\u003c/a\u003e\n    \u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\n  \u003cbr\u003e\n\u003c/div\u003e\n\n# Powered By [`⇧`](#contents)\n\n### Contributors\n\nMany thanks to all the [Scribe-Server contributors](https://github.com/scribe-org/Scribe-Server/graphs/contributors)! 🚀\n\n\u003ca href=\"https://github.com/scribe-org/Scribe-Server/graphs/contributors\"\u003e\n  \u003cimg src=\"https://contrib.rocks/image?repo=scribe-org/Scribe-Server\" /\u003e\n\u003c/a\u003e\n\n### Code and Dependencies\n\nThe Scribe community would like to thank all the great software that made Scribe-Server's development possible.\n\n### Wikimedia Communities\n\n\u003cdiv align=\"center\"\u003e\n  \u003cbr\u003e\n    \u003ca href=\"https://www.wikidata.org/\"\u003e\n      \u003cimg width=\"240\" src=\"https://raw.githubusercontent.com/scribe-org/Organization/main/resources/images/logos/WikidataLogo.png\" alt=\"Wikidata logo\"\u003e\n    \u003c/a\u003e\n    \u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\n    \u003ca href=\"https://www.wikipedia.org/\"\u003e\n      \u003cimg width=\"160\" src=\"https://raw.githubusercontent.com/scribe-org/Organization/main/resources/images/logos/WikipediaLogo.png\" alt=\"Wikipedia logo\"\u003e\n    \u003c/a\u003e\n    \u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\n    \u003ca href=\"https://admin.toolforge.org/\"\u003e\n      \u003cimg width=\"175\" src=\"https://raw.githubusercontent.com/scribe-org/Organization/main/resources/images/logos/WikimediaToolforgeLogo.png\" alt=\"Wikimedia Toolforge logo\"\u003e\n    \u003c/a\u003e\n    \u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\n  \u003cbr\u003e\n\u003c/div\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscribe-org%2Fscribe-server","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fscribe-org%2Fscribe-server","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscribe-org%2Fscribe-server/lists"}