{"id":24901077,"url":"https://github.com/eunomatix/llminspect-gateway","last_synced_at":"2026-02-04T01:41:49.981Z","repository":{"id":274332196,"uuid":"922592734","full_name":"eunomatix/llminspect-gateway","owner":"eunomatix","description":"LLMInspect AI Gateway is a cutting edge platforms to protect businesses from GenAI Threats","archived":false,"fork":false,"pushed_at":"2025-01-26T19:12:10.000Z","size":3209,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-07-07T03:10:52.553Z","etag":null,"topics":["ai","anthropic","chatgpt","cybersecurity","data-science","eunomatix","firewall","gemini","genai","generative-ai","guardrails","machine-learning","privacy","security-operations-center","threat-hunting","threat-intelligence"],"latest_commit_sha":null,"homepage":"https://eunomatix.com/site/web/products-llminspect.html","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/eunomatix.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-01-26T16:15:03.000Z","updated_at":"2025-01-26T20:07:38.000Z","dependencies_parsed_at":"2025-01-26T19:29:19.141Z","dependency_job_id":null,"html_url":"https://github.com/eunomatix/llminspect-gateway","commit_stats":null,"previous_names":["eunomatix/lllminspect-description","eunomatix/llminspect-gateway"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/eunomatix/llminspect-gateway","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eunomatix%2Fllminspect-gateway","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eunomatix%2Fllminspect-gateway/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eunomatix%2Fllminspect-gateway/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eunomatix%2Fllminspect-gateway/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/eunomatix","download_url":"https://codeload.github.com/eunomatix/llminspect-gateway/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eunomatix%2Fllminspect-gateway/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29063974,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-04T00:26:14.114Z","status":"ssl_error","status_checked_at":"2026-02-04T00:23:06.435Z","response_time":96,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","anthropic","chatgpt","cybersecurity","data-science","eunomatix","firewall","gemini","genai","generative-ai","guardrails","machine-learning","privacy","security-operations-center","threat-hunting","threat-intelligence"],"created_at":"2025-02-01T21:14:59.469Z","updated_at":"2026-02-04T01:41:49.961Z","avatar_url":"https://github.com/eunomatix.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"resources/llminspect.png\" width=\"328\"/\u003e\n\u003c/div\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://skillicons.dev\"\u003e\n    \u003cimg src=\"https://skillicons.dev/icons?i=sass,docker,aws,gcp,openshift,kubernetes,redhat,ubuntu,windows\u0026theme=light\" /\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\n## LLMInspect - An AI Gateway\nWith the recent rise of GenAI, most of the enterprises want to use LLMs to enhance their productivity, by enabling their employees to grant them access to public LLM providers (like **ChatGPT**, **Gemini**, **Anthropic**, etc). Though this definitely improves the overall productivity of the employees in their routine operations, however this capability posses several threats to the enterprise as mployees are sharing the intellectual property of the organization with these public entities, and thus opening a new avenue of data leakage threats.\n\n**LLMInspect** GenAI Gateway from \u003ca href=\"https://eunomatix.com\"\u003e**EUNOMATIX**\u003c/a\u003e is a one stop solution to all those problems, as it massively enhances the enterprise AI experience and cyber safety. From one application users can interact with different public and private LLM models, and control what is being sent to these models. Using **LLMInspect**, SOC Analysts can also view the employee GenAI usage and discover any LLM related Security Threats. **LLMInspect** also brings very interesting capabilities to its users, e.g. rate limit the GenAI queries or even re-route external LLM queries to enterprise's private LLMs.\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"resources/aisplash.png\" width=\"314\"/\u003e\n\u003c/div\u003e\n\n## Architecture\nLLMinspect is designed from the ground up keeping in mind the requirements of modern enterprise deployments. Based on the principles of **service-oriented architecture**, LLMinspect can be comfortably deployed both in the cloud and on-prem, and easily scalable on emerging business needs. LLMinspect enables clients from various modalities to interact with public LLM providers in a safe and secure manner such as chat clients using our flexible web interface which allows a user to fully exercise various features of LLMs for example conversational AI image generation, code generation and retrieval augmented generation. Regardless of the source all LLM interactions are guarded and logged by LLMInspect. \n\nOnce the requests reach LLMInspect, LLMinspect masks all **personally identifiable information (PII)** so that such information will not leave the enterprise premises in parallel our sentiment analysis service detects and logs the sentiment of the text. During this process detailed logs, chat text, images as well as transaction metadata are kept for analysis in the InspectDB service. \n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"resources/llminspect-architecture.png\" width=\"1024\"/\u003e\u003cbr\u003e\u003cbr\u003e\n\u003c/div\u003e\n\n## Features\n- **Simple GUI**: ChatGPT like GUI including Light and Dark Modes\n- **Multimodal Chat**: Upload and analyze images and leverage advanced agents with tools and API actions\n- **Multilingual UI**: Support for multiple languages\n- **Model Selection**: OpenAI, Azure, Anthropic, Google, and more\n- **Multi-User Support**: Integrates with LDAP, Active Directory and OpenID\n- **Flexible Deployment**: Proxy, Reverse Proxy, Docker, etc\n- **Searching**: Search all Messages and conversations\n- **Export**: Conversations in various formats (Markdown, JSON, etc)\n\n\n\n## Integration with Retrieval Augmented Generation (RAG)\nLLMinspect AI Gatways also includes an embedded indentity-aware \u003ca href=\"https://eunomatix.com/site/web/products-inspectrag.html\"\u003e**SecureRAG**\u003c/a\u003e, where users can upload custom documents to enhance an LLMs response or process. SecureRAG is quite intelligent and supports integration with enterprise data repositories (e.g. **Sharepoint**, **Google Drive**, etc.) in conjunction with the enterprise identity provider (**IdP**) to ensure that document access is safe, secure and only authorized users are able to access their respective documents in the RAG store.\n\n## Security Guardrails\nLLMInspect guardrails refer to safety mechanisms or guidelines put in place to control, constrain, or guide the behavior of large language models like ChatGPT or Gemini, ensuring they are used responsibly and ethically. These guardrails help prevent harmful outputs, mitigate risks of misinformation, and ensure that interactions are safe, aligned with ethical standards, and useful.\n\n- **Negative Sentiment Guardails**\n- **PII \u0026 BII Guardrail**\n- **Anti-Jailbreak Guardrail**\n- **Prompt Classifier Guardrail**\n- **Business-Specific Guardrails**\n\n## Advanced Features\nTo increase observability effectiveness LLMInspect provides plugins for easy integration with splunk and elastic. These plugins reduced the need for custom tooling and seamlessly integrate with existing SOC (security operation center) solutions. SIEM dashboards provide by LLMinspect enable SOC analysts to analyze this information. Finally, the masked text is sent to the upstream LLM of user’s choice.    \n\n## Contact us\nReach out LLLMInspect Sales Teams for free demo/trail, by emailing [sales@eunomatix.com](mailto:sales@eunomatix.com)\n\n[![CC BY 4.0][cc-by-shield]][cc-by]\n\nThis work is licensed under a\n[Creative Commons Attribution 4.0 International License][cc-by].\n\n[![CC BY 4.0][cc-by-image]][cc-by]\n\n[cc-by]: http://creativecommons.org/licenses/by/4.0/\n[cc-by-image]: https://i.creativecommons.org/l/by/4.0/88x31.png\n[cc-by-shield]: https://img.shields.io/badge/License-CC%20BY%204.0-lightgrey.svg\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feunomatix%2Fllminspect-gateway","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Feunomatix%2Fllminspect-gateway","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feunomatix%2Fllminspect-gateway/lists"}