https://github.com/zcemycl/qa-chatgpt-hf-pgvector
E-commerce fashion assistant with Chatgpt, Hugging Face, Ltree and Pgvector.
https://github.com/zcemycl/qa-chatgpt-hf-pgvector
captioning-images chatgpt cosine-similarity docker-compose dockerfile embeddings guardrails huggingface huggingface-transformers image-to-text jupyter-notebook ltree openai pgvector postgresql python sqlalchemy
Last synced: 8 months ago
JSON representation
E-commerce fashion assistant with Chatgpt, Hugging Face, Ltree and Pgvector.
- Host: GitHub
- URL: https://github.com/zcemycl/qa-chatgpt-hf-pgvector
- Owner: zcemycl
- Created: 2023-11-07T23:44:09.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2023-11-15T23:01:08.000Z (almost 2 years ago)
- Last Synced: 2025-02-17T10:53:18.556Z (8 months ago)
- Topics: captioning-images, chatgpt, cosine-similarity, docker-compose, dockerfile, embeddings, guardrails, huggingface, huggingface-transformers, image-to-text, jupyter-notebook, ltree, openai, pgvector, postgresql, python, sqlalchemy
- Language: Python
- Homepage:
- Size: 2.2 MB
- Stars: 3
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# qa-chatgpt-pgvector
|1. Produce Advice|2. Customer Conversation|
|---|---|
|
|
|
|3. Similar Garments Image Search|4. Complementarity Garments Suggestion|
|
|
|
## Schedule (8-15 Nov)
1. docker-compose.yml + Dockerfile.pgvector -> 8 Nov
- initialise a database with vector support.
1. test-openai.ipynb
- test openai==1.1.1 api, including embeddings and chat completions.
2. create-embeddings.ipynb
- create embeddings for all articles.
- discover 3 nan details in record.
- store embeddings in postgres using pgvector.
3. doc-search.ipynb
- prove that pgvector can help cosine similiarity calculation.
- only embedding details from dataframe are not enough as answers can be inaccurate.
- try question-answering with given information.
4. chatbot.ipynb
- implement chat completion loop.
5. guardrail_openai-0-28-1.ipynb
- try guardrails-ai.
- switch openai version to 0.28.x as 1.1.x is not compatible with guardrails-ai.
- try all embeddings and chat completions in 0.28.x openai api.
6. qa_package.main (Mode 1 and 2) -> 9 Nov
- create chatbot.
7. hf-captions.ipynb -> 12 Nov
- test gpt4 vision, it requires payment to use. (rejected)
- test hugging face Salesforce/blip-image-captioning-base model for captioning.
- analyse if clusterings can help suggestion in same or different categories.
8. qa_package.main (Mode 3) -> 13 Nov
- find garments based on text + image.
9. ltree.ipynb
- test ltree in postgres to group product groups and types.
- create color embedding
- create pattern embedding
- create garment embedding + setn.{product group}.{product type} as ltree path.
10. qa_package.main (Mode 4) -> 15 Nov
- suggest complementarity garments based on text + image.
## How to run?
1. Edit environment variables.
```
cp .env.example .env
cp .env jpnotes/
# Then fill in .env variables
```
1. Install required packages. `pip install -e .`
2. Initialise postgres in Docker. `docker compose up --build`
3. Run chatbot. (Only run with `--initialise-embeddings` for the first time)
```python
python -m qa_package.main \
--batch-size int \
--root-image-dir str \
--article-csv str \
--initialise-embeddings \
--visualise
```
for example,
```
python -m qa_package.main \
--batch-size 16 \
--initialise-embeddings \
--article-csv /Users/spare/Documents/data/articles.csv \
--root-image-dir /Users/spare/Documents/data/images/ \
--visualise
```
## References
1. https://pypi.org/project/openai/0.28.1/
- Old Documentation of openai 0.28.1
2. https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/switching-endpoints
- openai 0.28.1 vs openai 1.1.1
3. https://www.kommunicate.io/blog/create-a-customer-service-chatbot-using-chatgpt/
- Conversation loop for chatbot.
4. https://www.mlq.ai/fine-tuning-gpt-3-question-answer-bot/
- Question Answering with given information.
5. https://docs.guardrailsai.com/defining_guards/pydantic/
- Define Guardrails with Pydantic.
6. https://docs.guardrailsai.com/guardrails_ai/getting_started/#creating-a-rail-spec
- Guardrails example.
7. https://cookbook.openai.com/examples/gpt_with_vision_for_video_understanding
- openai 0.28.1 can support gpt4 vision preview.
8. https://huggingface.co/tasks/image-to-text
- hugging face image caption model
9. https://www.kite.com/blog/python/sqlalchemy/
- ltree operation in sqlalchemy-utils
10. https://www.postgresql.org/docs/current/ltree.html
- ltree type