Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/satendrakumar/openlit-fastapi-poc
openlit integration with FastAPI using Ollama model server
https://github.com/satendrakumar/openlit-fastapi-poc
docker-compose fastapi openlit opentelemetry oveservability poetry python
Last synced: about 7 hours ago
JSON representation
openlit integration with FastAPI using Ollama model server
- Host: GitHub
- URL: https://github.com/satendrakumar/openlit-fastapi-poc
- Owner: satendrakumar
- Created: 2025-01-21T07:29:35.000Z (about 21 hours ago)
- Default Branch: main
- Last Pushed: 2025-01-21T08:22:39.000Z (about 20 hours ago)
- Last Synced: 2025-01-21T09:24:54.987Z (about 19 hours ago)
- Topics: docker-compose, fastapi, openlit, opentelemetry, oveservability, poetry, python
- Language: Python
- Homepage:
- Size: 104 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# openlit-fastapi-poc
#### Start openlit server using docker compose:
```shell
cd openlit-docker
docker compose up -d
# Open browser:
http://localhost:3000/
username - [email protected]
password - openlituser
```
#### Ollama setup on local:
```shell
docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
docker exec -it ollama ollama run llama3.1
```
### Start openlit fastapi app:
```shell
poetry config virtualenvs.in-project true
poetry install
python main.py
### API docs
http://localhost:8000/docs
```
#### Test using curl request:
```shell
curl --location 'http://localhost:8000/v1/api/chat' \
--header 'Content-Type: application/json' \
--data '{
"model": "llama3",
"question": "Tell me about Nvidia'\''s Future?"
}'
#### response:
{
"answer": "NVIDIA is a leading technology company that has been at the forefront of innovation in various fields, including graphics processing units (GPUs), high-performance computing, artificial intelligence (AI), and autonomous vehicles. Here are some potential future developments and trends related to NVIDIA:\n\n**Short-term (2023-2025)**\n\n1. **Advancements in AI**: NVIDIA will continue to push the boundaries of AI research and development, with a focus on areas like natural language processing, computer vision, and reinforcement learning.\n2. **Increased adoption of GPUs for HPC**: As high-performance computing (HPC) becomes more widespread, NVIDIA's GPUs are expected to play an increasingly important role in accelerating simulations, modeling, and data analysis.\n3. **Expansion of autonomous vehicle technology**: NVIDIA will continue to develop its Drive platform, which enables the development of Level 2-5 autonomous vehicles.\n\n**Mid-term (2025-2030)**\n\n1. **Quantum Computing**: NVIDIA is exploring the potential of quantum computing and has already demonstrated a proof-of-concept for a quantum-accelerated GPU.\n2. **Advancements in Edge AI**: As edge computing becomes more prevalent, NVIDIA will focus on developing more efficient and powerful edge AI solutions that can be deployed at the edge of networks.\n3. **Increased adoption of GPUs in data centers**: With the growing demand for cloud services, NVIDIA's GPUs are expected to become even more ubiquitous in data centers.\n\n**Long-term (2030-2040)**\n\n1. **Neuromorphic Computing**: NVIDIA is researching neuromorphic computing architectures that mimic the human brain's neural networks. This could lead to significant breakthroughs in AI and machine learning.\n2. **Advancements in 3D XPoint Technology**: NVIDIA has acquired a company called Bright, which specializes in 3D XPoint technology (a type of non-volatile memory). This could enable faster and more efficient data storage solutions.\n3. **Increased focus on sustainability**: As the world becomes increasingly aware of environmental concerns, NVIDIA may prioritize developing more energy-efficient and sustainable technologies.\n\n**Speculative Future Developments**\n\n1. **Brain-Computer Interfaces (BCIs)**: NVIDIA has already demonstrated a BCI prototype that enables people to control devices with their thoughts.\n2. **Quantum-Accelerated AI**: As quantum computing advances, it's possible that NVIDIA will develop quantum-accelerated AI solutions that can solve complex problems in fields like medicine and climate modeling.\n3. **Holographic Displays**: With the development of advanced display technologies, NVIDIA may explore the creation of holographic displays that enable immersive experiences.\n\nKeep in mind that these are just predictions based on current trends and NVIDIA's past innovations. The future is inherently uncertain, and actual developments may differ from these projections."
}
```