https://github.com/kevinknights29/llama-v2-text-gen-api

This project aims to develop an API backend using Flask to support Text Generation Inference with Llama v2 Models
https://github.com/kevinknights29/llama-v2-text-gen-api

api-rest docker flask llama-2

Last synced: 23 days ago
JSON representation

This project aims to develop an API backend using Flask to support Text Generation Inference with Llama v2 Models

• Host: GitHub
• URL: https://github.com/kevinknights29/llama-v2-text-gen-api
• Owner: kevinknights29
• License: mit
• Created: 2023-07-23T11:21:45.000Z (over 1 year ago)
• Default Branch: main
• Last Pushed: 2024-03-02T05:09:31.000Z (8 months ago)
• Last Synced: 2024-05-08T00:23:59.409Z (6 months ago)
• Topics: api-rest, docker, flask, llama-2
• Language: Python
• Homepage:
• Size: 43 KB
• Stars: 1
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# LLama-v2-Text-Gen-API

This project aims to develop an API backend using Flask to support Text Generation Inference with Llama v2 Models

## Results

### Swagger-UI

![image](https://github.com/kevinknights29/LLama-v2-Text-Gen-API/assets/74464814/b8cc209c-4724-4201-ad7f-50338e15999c)

#### Text Generation Endpoint

![image](https://github.com/kevinknights29/LLama-v2-Text-Gen-API/assets/74464814/c96169ab-54ab-45d8-b764-f8e63d5d0fb1)

### API Call and Response

![image](https://github.com/kevinknights29/LLama-v2-Text-Gen-API/assets/74464814/708024d8-e64c-438b-b277-1cb511f61398)

### Streaming API

![image](https://github.com/kevinknights29/LLama-v2-Text-Gen-API/assets/74464814/14863e20-e840-477c-87aa-b874d6dd0b22)

## Usage

### Build APP Image

```bash

docker compose build

```

### Get everything up and running

```bash

docker compose down && docker compose up -d

```

### Have fun

Visit [http://localhost:7860/swagger](http://localhost:7860/swagger) to access the Swagger API Docs.

## Contributing

### Installing pre-commit

Pre-commit is already part of this project dependencies.

If you would like to install it as a standalone run:

```console

pip install pre-commit

```

To activate pre-commit run the following commands:

- Install Git hooks:

```console

pre-commit install

```

- Update current hooks:

```console

pre-commit autoupdate

```

To test your installation of pre-commit run:

```console

pre-commit run --all-files

```