https://github.com/ajndkr/lanarky
The web framework for building LLM microservices
https://github.com/ajndkr/lanarky
fastapi llmops microservices python3 web
Last synced: 3 days ago
JSON representation
The web framework for building LLM microservices
- Host: GitHub
- URL: https://github.com/ajndkr/lanarky
- Owner: ajndkr
- License: mit
- Created: 2023-04-07T05:36:32.000Z (about 2 years ago)
- Default Branch: main
- Last Pushed: 2024-07-06T06:21:57.000Z (9 months ago)
- Last Synced: 2025-04-03T13:15:17.129Z (11 days ago)
- Topics: fastapi, llmops, microservices, python3, web
- Language: Python
- Homepage: https://lanarky.ajndkr.com/
- Size: 22.9 MB
- Stars: 988
- Watchers: 15
- Forks: 76
- Open Issues: 9
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
Awesome Lists containing this project
- awesome-langchain-zh - Lanarky
- StarryDivineSky - ajndkr/lanarky
- awesome-langchain - Lanarky - ready LLM projects with FastAPI  (Tools / Services)
- awesome-llmops - lanarky - grade LLM applications |  | (Serving / Frameworks/Servers for Serving)
README
The web framework for building LLM microservices.
[](https://github.com/ajndkr/lanarky/stargazers)
[](https://github.com/ajndkr/lanarky/blob/main/LICENSE)
[](https://twitter.com/intent/follow?screen_name=LanarkyAPI)
[](https://pypi.org/project/lanarky/)
[](https://coveralls.io/github/ajndkr/lanarky?branch=main)
[](https://pypi.org/project/lanarky/)
[](https://pypistats.org/packages/lanarky)
> ⚠️ **Disclaimer**: This project is now in maintenance mode. I won't be adding new features or actively maintaining the project as I have moved on to other projects and priorities. While I will address critical bugs and security issues as needed, active development has ceased from my end. I do encourage the community to continue to contribute to the project if they find it useful. Thank you for using lanarky!
Lanarky is a **python (3.9+)** web framework for developers who want to build microservices using LLMs.
Here are some of its key features:
- **LLM-first**: Unlike other web frameworks, lanarky is built specifically for LLM developers.
It's unopinionated in terms of how you build your microservices and guarantees zero vendor lock-in
with any LLM tooling frameworks or cloud providers
- **Fast & Modern**: Built on top of FastAPI, lanarky offers all the FastAPI features you know and love.
If you are new to FastAPI, visit [fastapi.tiangolo.com](https://fastapi.tiangolo.com) to learn more
- **Streaming**: Streaming is essential for many real-time LLM applications, like chatbots. Lanarky has
got you covered with built-in streaming support over **HTTP** and **WebSockets**.
- **Open-source**: Lanarky is open-source and free to use. Forever.
To learn more about lanarky and get started, you can find the full documentation on [lanarky.ajndkr.com](https://lanarky.ajndkr.com)
## Installation
The library is available on PyPI and can be installed via `pip`:
```bash
pip install lanarky
```
## Getting Started
Lanarky provides a powerful abstraction layer to allow developers to build
simple LLM microservices in just a few lines of code.
Here's an example to build a simple microservice that uses OpenAI's `ChatCompletion` service:
```python
from lanarky import Lanarky
from lanarky.adapters.openai.resources import ChatCompletionResource
from lanarky.adapters.openai.routing import OpenAIAPIRouter
app = Lanarky()
router = OpenAIAPIRouter()
@router.post("/chat")
def chat(stream: bool = True) -> ChatCompletionResource:
system = "You are a sassy assistant"
return ChatCompletionResource(stream=stream, system=system)
app.include_router(router)
```
Visit [Getting Started](https://lanarky.ajndkr.com/getting-started) for the full tutorial on building
and testing your first LLM microservice with Lanarky.
## Contributing
[](https://github.com/ajndkr/lanarky/actions/workflows/code-check.yaml)
[](https://github.com/ajndkr/lanarky/actions/workflows/publish.yaml)
Contributions are more than welcome! If you have an idea for a new feature or want to help improve lanarky,
please create an issue or submit a pull request on [GitHub](https://github.com/ajndkr/lanarky).
See [CONTRIBUTING.md](https://github.com/ajndkr/lanarky/blob/main/CONTRIBUTING.md) for more information.
See [Lanarky Roadmap](https://github.com/users/ajndkr/projects/6) for the list of new features and future milestones.
## License
The library is released under the [MIT License](https://github.com/ajndkr/lanarky/blob/main/LICENSE).