{"id":32263041,"url":"https://github.com/mihaivalentin/lunr-languages","last_synced_at":"2025-10-22T20:46:57.665Z","repository":{"id":16220293,"uuid":"18967515","full_name":"MihaiValentin/lunr-languages","owner":"MihaiValentin","description":"A collection of languages stemmers and stopwords for Lunr Javascript library","archived":false,"fork":false,"pushed_at":"2025-03-09T18:59:44.000Z","size":1135,"stargazers_count":447,"open_issues_count":50,"forks_count":166,"subscribers_count":18,"default_branch":"master","last_synced_at":"2025-10-18T21:56:38.207Z","etag":null,"topics":["language-stemmer","localization","lunr","lunr-languages","stemmer","stopwords"],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MihaiValentin.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2014-04-20T15:41:13.000Z","updated_at":"2025-10-17T02:11:05.000Z","dependencies_parsed_at":"2024-08-01T12:23:51.869Z","dependency_job_id":"867b2802-4134-421c-9ce2-b868d405dc60","html_url":"https://github.com/MihaiValentin/lunr-languages","commit_stats":{"total_commits":115,"total_committers":24,"mean_commits":4.791666666666667,"dds":0.7130434782608696,"last_synced_commit":"24f03a2c2e0652c47fc6f5416f2e4619a14918de"},"previous_names":[],"tags_count":19,"template":false,"template_full_name":null,"purl":"pkg:github/MihaiValentin/lunr-languages","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MihaiValentin%2Flunr-languages","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MihaiValentin%2Flunr-languages/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MihaiValentin%2Flunr-languages/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MihaiValentin%2Flunr-languages/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MihaiValentin","download_url":"https://codeload.github.com/MihaiValentin/lunr-languages/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MihaiValentin%2Flunr-languages/sbom","scorecard":{"id":94106,"data":{"date":"2025-08-11","repo":{"name":"github.com/MihaiValentin/lunr-languages","commit":"190ad03ed756e51fba4a2907821e206e61a3b313"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":3.5,"checks":[{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Code-Review","score":5,"reason":"Found 8/15 approved changesets -- score normalized to 5","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Pinned-Dependencies","score":-1,"reason":"no dependencies found","details":null,"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"License","score":9,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Warn: project license file does not contain an FSF or OSI license."],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":-1,"reason":"internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration","details":null,"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 23 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}},{"name":"Vulnerabilities","score":5,"reason":"5 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: GHSA-v6h2-p8h4-qcjw","Warn: Project is vulnerable to: GHSA-grv7-fg5c-xmjg","Warn: Project is vulnerable to: GHSA-mwcw-c2x4-8c55","Warn: Project is vulnerable to: GHSA-c2qf-rxjj-qqgw","Warn: Project is vulnerable to: GHSA-76p7-773f-r4q5"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}}]},"last_synced_at":"2025-08-15T08:31:27.436Z","repository_id":16220293,"created_at":"2025-08-15T08:31:27.436Z","updated_at":"2025-08-15T08:31:27.436Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279952190,"owners_count":26249876,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-19T02:00:07.647Z","response_time":64,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["language-stemmer","localization","lunr","lunr-languages","stemmer","stopwords"],"created_at":"2025-10-22T20:46:57.025Z","updated_at":"2025-10-22T20:46:57.660Z","avatar_url":"https://github.com/MihaiValentin.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"Lunr Languages [![npm](https://img.shields.io/npm/v/lunr-languages.svg)](https://www.npmjs.com/package/lunr-languages) [![Bower](https://img.shields.io/bower/v/lunr-languages.svg)]() [![Join the chat at https://gitter.im/lunr-languages/Lobby](https://badges.gitter.im/lunr-languages/Lobby.svg)](https://gitter.im/lunr-languages/Lobby?utm_source=badge\u0026utm_medium=badge\u0026utm_campaign=pr-badge\u0026utm_content=badge) [![](https://img.shields.io/badge/compatible%20with%20Lunr-0.6.0%20--%3E%202.x-green.svg)](http://lunrjs.com/) [![CircleCI branch](https://img.shields.io/circleci/project/github/MihaiValentin/lunr-languages.svg)](https://circleci.com/gh/MihaiValentin/lunr-languages)\n==============\n\nLunr Languages is a [Lunr](http://lunrjs.com/) addon that helps you search in documents written in the following languages:\n\n* ![](https://raw.githubusercontent.com/madebybowtie/FlagKit/master/Assets/PNG/DE.png) German\n* ![](https://raw.githubusercontent.com/madebybowtie/FlagKit/master/Assets/PNG/FR.png) French\n* ![](https://raw.githubusercontent.com/madebybowtie/FlagKit/master/Assets/PNG/ES.png) Spanish\n* ![](https://raw.githubusercontent.com/madebybowtie/FlagKit/master/Assets/PNG/IT.png) Italian\n* ![](https://raw.githubusercontent.com/madebybowtie/FlagKit/master/Assets/PNG/NL.png) Dutch\n* ![](https://raw.githubusercontent.com/madebybowtie/FlagKit/master/Assets/PNG/DK.png) Danish\n* ![](https://raw.githubusercontent.com/madebybowtie/FlagKit/master/Assets/PNG/PT.png) Portuguese\n* ![](https://raw.githubusercontent.com/madebybowtie/FlagKit/master/Assets/PNG/FI.png) Finnish\n* ![](https://raw.githubusercontent.com/madebybowtie/FlagKit/master/Assets/PNG/RO.png) Romanian\n* ![](https://raw.githubusercontent.com/madebybowtie/FlagKit/master/Assets/PNG/HU.png) Hungarian\n* ![](https://raw.githubusercontent.com/madebybowtie/FlagKit/master/Assets/PNG/RU.png) Russian\n* ![](https://raw.githubusercontent.com/madebybowtie/FlagKit/master/Assets/PNG/NO.png) Norwegian\n* ![](https://raw.githubusercontent.com/madebybowtie/FlagKit/master/Assets/PNG/SE.png) Swedish\n* ![](https://raw.githubusercontent.com/madebybowtie/FlagKit/master/Assets/PNG/TR.png) Turkish\n* ![](https://raw.githubusercontent.com/madebybowtie/FlagKit/master/Assets/PNG/JP.png) Japanese\n* ![](https://raw.githubusercontent.com/madebybowtie/FlagKit/master/Assets/PNG/TH.png) Thai\n* ![](https://raw.githubusercontent.com/madebybowtie/FlagKit/master/Assets/PNG/IQ.png) Arabic\n* ![](https://raw.githubusercontent.com/madebybowtie/FlagKit/master/Assets/PNG/CN.png) Chinese\n* ![](https://raw.githubusercontent.com/madebybowtie/FlagKit/master/Assets/PNG/VN.png) Vietnamese\n* ![](https://raw.githubusercontent.com/madebybowtie/FlagKit/master/Assets/PNG/IN.png) Sankrit\n* ![](https://raw.githubusercontent.com/madebybowtie/FlagKit/master/Assets/PNG/IN.png) Kannada\n* ![](https://raw.githubusercontent.com/madebybowtie/FlagKit/master/Assets/PNG/IN.png) Telugu\n* ![](https://raw.githubusercontent.com/madebybowtie/FlagKit/master/Assets/PNG/IN.png) Hindi\n* ![](https://raw.githubusercontent.com/madebybowtie/FlagKit/master/Assets/PNG/IN.png) Tamil\n* ![](https://raw.githubusercontent.com/madebybowtie/FlagKit/master/Assets/PNG/KR.png) Korean\n* ![](https://raw.githubusercontent.com/madebybowtie/FlagKit/master/Assets/PNG/AM.png) Armenian\n* ![](https://raw.githubusercontent.com/madebybowtie/FlagKit/master/Assets/PNG/IL.png) Hebrew\n* ![](https://raw.githubusercontent.com/madebybowtie/FlagKit/master/Assets/PNG/GR.png) Greek\n* [Contribute with a new language](CONTRIBUTING.md)\n\nLunr Languages is compatible with Lunr version `0.6`, `0.7`, `1.0` and `2.X`.\n\n# How to use\n\nLunr-languages works well with script loaders (Webpack, requirejs) and can be used in the browser and on the server.\n\n## In a web browser\n\nThe following example is for the German language (de).\n\nAdd the following JS files to the page:\n\n```html\n\u003cscript src=\"lunr.js\"\u003e\u003c/script\u003e \u003c!-- lunr.js library --\u003e\n\u003cscript src=\"lunr.stemmer.support.js\"\u003e\u003c/script\u003e\n\u003cscript src=\"lunr.de.js\"\u003e\u003c/script\u003e \u003c!-- or any other language you want --\u003e\n```\n\nthen, use the language in when initializing lunr:\n\n```javascript\nvar idx = lunr(function () {\n  // use the language (de)\n  this.use(lunr.de);\n  // then, the normal lunr index initialization\n  this.field('title', { boost: 10 });\n  this.field('body');\n  // now you can call this.add(...) to add documents written in German\n});\n```\n\nThat's it. Just add the documents and you're done. When searching, the language stemmer and stopwords list will be the one you used.\n\n## In a web browser, with RequireJS\n\nAdd `require.js` to the page:\n\n```html\n\u003cscript src=\"lib/require.js\"\u003e\u003c/script\u003e\n```\n\nthen, use the language in when initializing lunr:\n\n```javascript\nrequire(['lib/lunr.js', '../lunr.stemmer.support.js', '../lunr.de.js'], function(lunr, stemmerSupport, de) {\n  // since the stemmerSupport and de add keys on the lunr object, we'll pass it as reference to them\n  // in the end, we will only need lunr.\n  stemmerSupport(lunr); // adds lunr.stemmerSupport\n  de(lunr); // adds lunr.de key\n\n  // at this point, lunr can be used\n  var idx = lunr(function () {\n  // use the language (de)\n  this.use(lunr.de);\n  // then, the normal lunr index initialization\n  this.field('title', { boost: 10 })\n  this.field('body')\n  // now you can call this.add(...) to add documents written in German\n  });\n});\n```\n\n# With node.js\n\n```javascript\nvar lunr = require('./lib/lunr.js');\nrequire('./lunr.stemmer.support.js')(lunr);\nrequire('./lunr.de.js')(lunr); // or any other language you want\n\nvar idx = lunr(function () {\n  // use the language (de)\n  this.use(lunr.de);\n  // then, the normal lunr index initialization\n  this.field('title', { boost: 10 })\n  this.field('body')\n  // now you can call this.add(...) to add documents written in German\n});\n```\n\n# Indexing multi-language content\n\nIf your documents are written in more than one language, you can enable multi-language indexing. This ensures every word is properly trimmed and stemmed, every stopword is removed, and no words are lost (indexing in just one language would remove words from every other one.)\n\n```javascript\nvar lunr = require('./lib/lunr.js');\nrequire('./lunr.stemmer.support.js')(lunr);\nrequire('./lunr.ru.js')(lunr);\nrequire('./lunr.multi.js')(lunr);\n\nvar idx = lunr(function () {\n  // the reason \"en\" does not appear above is that \"en\" is built in into lunr js\n  this.use(lunr.multiLanguage('en', 'ru'));\n  // then, the normal lunr index initialization\n  // ...\n});\n```\n\nYou can combine any number of supported languages this way. The corresponding lunr language scripts must be loaded (English is built in).\n\nIf you serialize the index and load it in another script, you'll have to initialize the multi-language support in that script, too, like this:\n\n```javascript\nlunr.multiLanguage('en', 'ru');\nvar idx = lunr.Index.load(serializedIndex);\n```\n\n# How to add a new language\n\nCheck the [Contributing](CONTRIBUTING.md) section\n\n# How does Lunr Languages work?\n\nSearching inside documents is not as straight forward as using `indexOf()`, since there are many things to consider in order to get quality search results:\n* **Tokenization**\n    * Given a string like *\"Hope you like using Lunr Languages!\"*, the tokenizer would split it into individual words, becoming an array like `['Hope', 'you', 'like', 'using', 'Lunr', 'Languages!']`\n    * Though it seems a trivial task for Latin characters (just splitting by the space), it gets more complicated for languages like Japanese. Lunr Languages has this included for the Japanese language.\n* **Trimming**\n    * After tokenization, trimming ensures that the words contain *just* what is needed in them. In our example above, the trimmer would convert `Languages!` into `Languages`\n    * So, the trimmer basically removes special characters that do not add value for the search purpose.\n* **Stemming**\n    * What happens if our text contains the word `consignment` but we want to search for `consigned`? It should find it, since its meaning is the same, only the form is different.\n    * A stemmer extracts the root of words that can have many forms and stores it in the index. Then, any search is also stemmed and searched in the index.\n    * Lunr Languages does stemming for all the included languages, so you can capture all the forms of words in your documents.\n* **Stop words**\n    * There's no point in adding or searching words like `the`, `it`, `so`, etc. These words are called *Stop words*\n    * Stop words are removed so your index will only contain meaningful words.\n    * Lunr Languages includes stop words for all the included languages.\n\n# Technical details \u0026 Credits\n\nI've created this project by compiling and wrapping stemmers toghether with stop words from various sources ([including users contributions](https://github.com/MihaiValentin/lunr-languages/pulls?q=is%3Apr)) so they can be directly used with all the current versions of Lunr.\n\n* \u003chttps://github.com/fortnightlabs/snowball-js\u003e (the stemmers for all languages, ported from snowball-js)\n* \u003chttps://github.com/brenes/stopwords-filter\u003e (the stop words list for the other languages)\n* \u003chttp://chasen.org/~taku/software/TinySegmenter/\u003e (the tinyseg Tiny Segmente Japanese tokenizer)\n\nI am providing code in the repository to you under an [open source license](LICENSE). Because this is my personal repository, the license you receive to my code is from me and not my employer (Facebook)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmihaivalentin%2Flunr-languages","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmihaivalentin%2Flunr-languages","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmihaivalentin%2Flunr-languages/lists"}