{"id":18729962,"url":"https://github.com/jovotech/snips-nlu-server","last_synced_at":"2025-04-12T17:00:25.017Z","repository":{"id":87061362,"uuid":"392380369","full_name":"jovotech/snips-nlu-server","owner":"jovotech","description":"An open source natural language understanding (NLU) server","archived":false,"fork":false,"pushed_at":"2022-11-14T12:16:21.000Z","size":31,"stargazers_count":17,"open_issues_count":2,"forks_count":3,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-04-11T17:21:25.657Z","etag":null,"topics":["nlp","nlu","nlu-engine","snips-nlu"],"latest_commit_sha":null,"homepage":"https://www.jovo.tech/marketplace/nlu-snips","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jovotech.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2021-08-03T16:24:04.000Z","updated_at":"2024-10-22T04:05:09.000Z","dependencies_parsed_at":"2023-09-24T16:57:19.595Z","dependency_job_id":null,"html_url":"https://github.com/jovotech/snips-nlu-server","commit_stats":{"total_commits":30,"total_committers":5,"mean_commits":6.0,"dds":"0.23333333333333328","last_synced_commit":"f4ad4625fe91a5544bb5d15aaa5be88d6a841e14"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jovotech%2Fsnips-nlu-server","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jovotech%2Fsnips-nlu-server/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jovotech%2Fsnips-nlu-server/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jovotech%2Fsnips-nlu-server/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jovotech","download_url":"https://codeload.github.com/jovotech/snips-nlu-server/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248602273,"owners_count":21131613,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["nlp","nlu","nlu-engine","snips-nlu"],"created_at":"2024-11-07T14:31:10.364Z","updated_at":"2025-04-12T17:00:24.949Z","avatar_url":"https://github.com/jovotech.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Snips NLU Server\n\nThis server allows you to run [Snips NLU](https://github.com/snipsco/snips-nlu), an open source [natural language understanding (NLU)](https://www.jovo.tech/docs/nlu) engine, on your own premises.\n\nUsing this server is recommended if you want to build [Jovo](https://www.jovo.tech) apps using the [Snips NLU integration](https://www.jovo.tech/marketplace/nlu-snips). It features [intent scoping](https://www.jovo.tech/marketplace/nlu-snips#intent-scoping) and even the ability to dynamically train the language model called [dynamic entities](#dynamic-entities).\n\nLearn more below:\n- [Setup](#setup)\n- [Training an Engine](#training-an-engine)\n- [Dynamic Entities](#dynamic-entities)\n- [Parsing a Message](#parsing-a-message)\n\n\n## Setup\n\nFirst, clone this repository:\n\n```sh\n$ git clone https://github.com/jovotech/snips-nlu-server.git\n\n$ cd snips-nlu-server\n```\n\nThere are two options to run the server:\n- [Using Docker](#using-docker) (recommended)\n- [Manual Setup](#manual-setup)\n\n### Using Docker\n\nThe easiest way to run the server is using Docker.\n\nYou can build the image yourself and run it on port 5000:\n\n```sh\ndocker build -t snips-nlu-server .\ndocker run -p 5001:5001 snips-nlu-server\n```\n\n### Manual Setup\n\nTo be able to interact with the Snips NLU engine, the server is implemented in Python. If you haven't already installed Python on your system, you can follow [this guide](https://realpython.com/installing-python/). There have been some issues with using Python \u003e 3.9, so we recommend sticking to either Python 2.7 or Python \u003e= 3.5 \u003c= 3.9.\n\nTo manage dependencies on a project-scoped level, you need to create a [virtual environment](https://docs.python.org/3/tutorial/venv.html) and activate it:\n\n```sh\n# Create a virtual environment\n$ python -m venv venv/\n\n# Activate the environment\n$ source venv/bin/activate\n```\n\nDepending on your shell you might need to run another script, read more [here](https://docs.python.org/3/tutorial/venv.html)\n\nTo deactivate/exit the virtual environment, run:\n\n```sh\n$ deactivate\n```\n\n#### Install Dependencies\n\nTo be able to use such libraries as `snips-nlu` and `flask`, you'll need to install all required dependencies specified in `requirements.txt`:\n\n```sh\n$ pip install -r requirements.txt\n```\n\nIf you get an error stating that the Rust compiler can't be found, try installing it using [this guide](https://rustup.rs/).\n\nSince the server uses `@jovotech/model-snips` for conversion, you need to install Node dependencies as well:\n\n```sh\n$ npm install\n```\n\nAdditionally, Snips NLU requires you to download your language resources manually:\n\n```sh\n$ python -m snips-nlu download \u003clanguage\u003e\n\n# or natively\n$ snips-nlu download \u003clanguage\u003e\n```\n\n#### Run your Server\n\nSet environment variables so Flask can find your server file:\n\n```sh\n# Linux/MacOS\n$ export FLASK_APP=server/__init__.py\n\n# Windows\n$ set FLASK_APP=server/__init__.py\n```\n\nIf you want to restart the server on changes automatically, add this to your environment:\n\n```sh\n# Linux/MacOS\n$ export FLASK_ENV=development\n\n# Windows\n$ set FLASK_ENV=development\n```\n\nStart the server using the following command:\n\n```sh\n$ python -m flask run\n\n# or natively\n$ flask run\n```\n\n## Training an Engine\n\nThe Snips NLU server provides an endpoint `/engine/train` that lets you train and persist a Snips NLU engine from a [Jovo Language Model](https://www.jovo.tech/docs/models). It accepts the following query parameters:\n\n- `locale`: The locale of the language model\n- `engine_id`: The ID which is used to persist the NLU engine\n\nYou can choose between providing the model directly in the request body as part of the `POST` request, or set an environment variable `MODEL_LOCATION`, which holds the endpoint to the containing directory of your model (e.g. S3). Note that this endpoint is joined with the locale provided in the request query, so the final model endpoint will look something like this: `${MODEL_LOCATION}/${locale}.json`.\n\nThe language model is then converted to a native Snips NLU dataset format, which can be further used to train the engine you'll later use to parse messages through. Using the provided query parameters `engine_id` and `locale`, this engine is then persisted locally to be reloaded when needed. This approach allows you to load your engine on demand, rather than to keep it running in a dedicated server instance. It also enables you to make use of [Dynamic Entities](https://www.jovo.tech/marketplace/nlu-snips#dynamic-entities), which will be explained below.\n\n## Dynamic Entities\n\nBy persisting the trained engines, it's possible to train a dedicated engine for dynamic entities on the fly and parse a message through it, if fit, otherwise the default engine is used. Note that in contrast to training a regular engine, this endpoint is called automatically by the [Snips NLU integration](https://www.jovo.tech/marketplace/nlu-snips), if you set `entities` in your output object. Learn more [here](https://www.jovo.tech/docs/entities#dynamic-entities).\n\nThe `POST` endpoint `/engine/train/dynamic-entities` accepts the following query parameters:\n\n- `locale`: The locale of the language model\n- `engine_id`: The ID which is used to persist the NLU engine\n- `entity`: The dynamic entity name\n- `session_id`: The current session ID\n\nApart from `engine_id`, which will be taken from [configuration](https://www.jovo.tech/marketplace/nlu-snips#configuration), these parameters will be set automatically. You can also set whether you want to pass the model automatically by setting `passModels` to `true` in the app configuration or provide your own models as described above.\n\nNote that if you provide your own model, it only needs to contain the dynamic entity, along with every updated intent containing the entity.\n\n## Parsing a Message\n\nThe main endpoint of the server is `/engine/parse`, which will be called by the [Snips NLU integration](https://www.jovo.tech/marketplace/nlu-snips) to parse a message and return the resolved values. It also includes parsing messages with dynamic entities. It accepts the following query parameters:\n\n- `engine_id`: The ID which is used to load the NLU engine\n- `session_id`: The current session ID\n- `locale`: The locale of the current Jovo request\n\nSimilarly to the other endpoints, those parameters are either taken from the app configuration or set automatically, depending on the incoming request.\nThe Snips NLU server then loads the previously trained and persisted engine and parses the message provided in the `POST` request body through it. The server will either use an engine trained for dynamic entities, if present, or fall back to the default engine.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjovotech%2Fsnips-nlu-server","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjovotech%2Fsnips-nlu-server","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjovotech%2Fsnips-nlu-server/lists"}