{"id":48931240,"url":"https://github.com/gravitee-io/gravitee-inference","last_synced_at":"2026-04-23T14:01:27.057Z","repository":{"id":288294463,"uuid":"967504287","full_name":"gravitee-io/gravitee-inference","owner":"gravitee-io","description":"A set of libraries to integrate ML/AI in gravitee projects","archived":false,"fork":false,"pushed_at":"2026-04-10T14:58:00.000Z","size":152,"stargazers_count":2,"open_issues_count":12,"forks_count":0,"subscribers_count":9,"default_branch":"main","last_synced_at":"2026-04-10T16:29:40.211Z","etag":null,"topics":["ai","inference","machine-learning","security-scan"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/gravitee-io.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-04-16T14:55:45.000Z","updated_at":"2026-02-10T08:42:30.000Z","dependencies_parsed_at":"2025-06-17T06:23:43.563Z","dependency_job_id":"831d3988-7b9d-46e3-ad72-95315ed3ed40","html_url":"https://github.com/gravitee-io/gravitee-inference","commit_stats":null,"previous_names":["gravitee-io/gravitee-inference"],"tags_count":9,"template":false,"template_full_name":null,"purl":"pkg:github/gravitee-io/gravitee-inference","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gravitee-io%2Fgravitee-inference","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gravitee-io%2Fgravitee-inference/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gravitee-io%2Fgravitee-inference/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gravitee-io%2Fgravitee-inference/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/gravitee-io","download_url":"https://codeload.github.com/gravitee-io/gravitee-inference/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gravitee-io%2Fgravitee-inference/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31922399,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-16T18:22:33.417Z","status":"online","status_checked_at":"2026-04-17T02:00:06.879Z","response_time":62,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","inference","machine-learning","security-scan"],"created_at":"2026-04-17T09:02:12.563Z","updated_at":"2026-04-17T09:02:13.253Z","avatar_url":"https://github.com/gravitee-io.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# gravitee-inference\n\n**gravitee-inference** is a Java library designed to make it easy for engineering teams to integrate and deploy AI models within the Gravitee platform—without needing specialized help from AI/ML teams.\n\n---\n\n## Requirements\n\n- Java 21\n- Maven (`mvn`)\n\n---\n\n## Import libraries\n\nIn your `pom.xml` add the dependencies\n\n```xml\n\u003cdependency\u003e\n  \u003cgroupId\u003eio.gravitee.inference.math.native\u003c/groupId\u003e\n  \u003cartifactId\u003egravitee-inference-math-native\u003c/artifactId\u003e\n  \u003cversion\u003e${gravitee.inference.version}\u003c/version\u003e\n\u003c/dependency\u003e\n\n\u003cdependency\u003e\n  \u003cgroupId\u003eio.gravitee.inference.api\u003c/groupId\u003e\n  \u003cartifactId\u003egravitee-inference-api\u003c/artifactId\u003e\n  \u003cversion\u003e${gravitee.inference.version}\u003c/version\u003e\n\u003c/dependency\u003e\n\n\u003cdependency\u003e\n  \u003cgroupId\u003eio.gravitee.inference.onnx\u003c/groupId\u003e\n  \u003cartifactId\u003egravitee-inference-onnx\u003c/artifactId\u003e\n  \u003cversion${gravitee.inference.version}\u003c/version\u003e\n\u003c/dependency\u003e\n```\n\n## Supported AI Models\n\n### BERT (via ONNX)\n\nWe support BERT architecture in ONNX format for various NLP tasks:\n\n- Sequence Classification\n- Token Classification\n- Fill-mask\n- Vector Embedding (e.g., Sentence Similarity)\n\n---\n\n### 🧠 Sequence Classification\n\nUse this to determine sentiment or categorize full sentences.\n\n```java\nvar resource = new OnnxBertResource(\n    Paths.get(\"/path/to/your/model.onnx\"),\n    Paths.get(\"/path/to/your/tokenizer.json\")\n);\n\nvar configuration = Map.of(\n    CLASSIFIER_MODE, ClassifierMode.SEQUENCE,\n    CLASSIFIER_LABELS, List.of(\"Negative\", \"Positive\")\n);\n\nvar onnxConfig = new OnnxBertConfig(\n    resource,\n    NativeMath.INSTANCE,\n    configuration\n);\n\nvar model = new OnnxBertClassifierModel(onnxConfig);\n\n// Single sentence\nList\u003cClassifierResult\u003e results = model.infer(\"I am so happy!\").results();\nresults.forEach(result -\u003e {\n    System.out.println(\"Label: \" + result.label());\n    System.out.println(\"Score: \" + result.score());\n});\n\n// Multiple sentences\nmodel.infer(List.of(\"I am so happy!\", \"I am so sad!\"));\n```\n\n\u003e Try this with [`distilbert-base-uncased-finetuned-sst-2-english`](https://huggingface.co/distilbert/distilbert-base-uncased-finetuned-sst-2-english).\n\n---\n\n### 🧾 Token Classification\n\nUse this to extract structured entities like names, locations, and organizations from text.\n\n```java\nvar resource = new OnnxBertResource(\n    Paths.get(\"/path/to/your/model.onnx\"),\n    Paths.get(\"/path/to/your/tokenizer.json\")\n);\n\nvar configuration = Map.of(\n    Constants.CLASSIFIER_MODE, ClassifierMode.TOKEN,\n    Constants.CLASSIFIER_LABELS, List.of(\n        \"O\", \"B-MISC\", \"I-MISC\", \"B-PER\", \"I-PER\", \"B-ORG\", \"I-ORG\", \"B-LOC\", \"I-LOC\"\n    ),\n    Constants.DISCARD_LABELS, List.of(\"O\", \"B-MISC\", \"I-MISC\")\n);\n\nvar onnxConfig = new OnnxBertConfig(resource, NativeMath.INSTANCE, configuration);\nvar model = new OnnxBertClassifierModel(onnxConfig);\n\nList\u003cClassifierResult\u003e results = model.infer(\"My name is Laura and I live in Houston, Texas\").results();\nresults.forEach(result -\u003e {\n    System.out.println(\"Label: \" + result.label());\n    System.out.println(\"Score: \" + result.score());\n    System.out.println(\"Begin: \" + result.begin());\n    System.out.println(\"End: \" + result.end());\n});\n```\n\n```java\nmodel.infer(List.of(\n    \"My name is Laura and I live in Houston, Texas\",\n    \"My name is Clara and I live in Berkley, California\"\n));\n```\n\n\u003e Try this with [`dslim/bert-base-NER`](https://huggingface.co/dslim/bert-base-NER/).\n\n---\n\n### 🎭 Fill Mask\n\nPredict masked tokens in a sentence.\n\n```java\nvar resource = new OnnxBertResource(\n    Paths.get(\"/path/to/your/model.onnx\"),\n    Paths.get(\"/path/to/your/tokenizer.json\")\n);\n\nvar onnxConfig = new OnnxBertConfig(resource, NativeMath.INSTANCE, Map.of());\nvar model = new OnnxBertFillMaskInference(onnxConfig);\n\nList\u003cFillMaskResult\u003e results = model.infer(\"The capital of France is [MASK].\");\n\nSystem.out.println(results.getFirst().label()); // Paris\n```\n\n```java\nmodel.infer(List.of(\n    \"The capital of France is [MASK].\",\n    \"The capital of [MASK] is London.\"\n));\n```\n\n\u003e Try this with [`google-bert/bert-base-uncased`](https://huggingface.co/google-bert/bert-base-uncased).\n\n---\n\n### 📐 Vector Embeddings\n\nConvert text into dense vector representations for similarity search or indexing.\n\n```java\nvar resource = new OnnxBertResource(\n    Paths.get(\"/path/to/your/model.onnx\"),\n    Paths.get(\"/path/to/your/tokenizer.json\")\n);\n\nvar onnxConfig = new OnnxBertConfig(resource, NativeMath.INSTANCE, Map.of(\n    POOLING_MODE, PoolingMode.MEAN,\n    Constants.MAX_SEQUENCE_LENGTH, 512\n));\n\nvar model = new OnnxBertEmbeddingModel(onnxConfig);\nEmbeddingTokenCount embedding = model.infer(\"The big brown fox jumped over the lazy dog\");\n\nSystem.out.println(embedding.embedding().length); // 384\nSystem.out.println(embedding.tokenCount()); // 11\n\n// Similarity comparison\nEmbeddingTokenCount embedding1 = model.infer(\"The big brown fox jumped over the lazy dog\");\nEmbeddingTokenCount embedding2 = model.infer(\"The brown fox jumped over the dog\");\n\nSystem.out.println(\n    onnxConfig.gioMaths().cosineScore(embedding1.embedding(), embedding2.embedding())\n);\n```\n\n\u003e Try this with [`Xenova/all-MiniLM-L6-v2`](https://huggingface.co/Xenova/all-MiniLM-L6-v2).\n\n---\n\n### ⚡ SIMD Capabilities\n\nTo run with SIMD math acceleration:\n\n1. Add the following to your JVM arguments:\n\n```sh\n--add-modules jdk.incubator.vector\n```\n\n2. Import the according dependencies:\n\n```xml\n\u003cdependency\u003e\n    \u003cgroupId\u003eio.gravitee.inference.math.simd\u003c/groupId\u003e\n    \u003cartifactId\u003egravitee-inference-math-simd\u003c/artifactId\u003e\n    \u003cversion\u003e${gravitee.inference.version}\u003c/version\u003e\n\u003c/dependency\u003e\n```\n\n```java\nimport io.gravitee.inference.math.simd.factory.SIMDMathFactory;\n\nGioMaths maths = SIMDMathFactory.gioMaths();\n```\n\nThe factory will resolve at runtime which SIMD capability your CPU handles.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgravitee-io%2Fgravitee-inference","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgravitee-io%2Fgravitee-inference","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgravitee-io%2Fgravitee-inference/lists"}