https://github.com/aws-samples/rds-postgresql-pgvector

https://github.com/aws-samples/rds-postgresql-pgvector

Last synced: 9 days ago
JSON representation

• Host: GitHub
• URL: https://github.com/aws-samples/rds-postgresql-pgvector
• Owner: aws-samples
• License: mit-0
• Created: 2023-05-01T19:10:20.000Z (over 1 year ago)
• Default Branch: main
• Last Pushed: 2023-08-28T14:46:28.000Z (about 1 year ago)
• Last Synced: 2024-08-02T18:40:36.345Z (4 months ago)
• Language: Jupyter Notebook
• Size: 2.04 MB
• Stars: 31
• Watchers: 3
• Forks: 7
• Open Issues: 2
• Metadata Files:
  • Readme: README.md
  • Contributing: CONTRIBUTING.md
  • License: LICENSE
  • Code of conduct: CODE_OF_CONDUCT.md

Awesome Lists containing this project

README

        
# Building AI-powered search in PostgreSQL using Amazon SageMaker and pgvector

This repository guides users through creating a product similarity search using Amazon SageMaker and Amazon RDS for PostgreSQL using the extension `pgvector`.

# How does it work?

we have used pre-trained model [`all-MiniLM-L6-v2`](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) from Hugging Face SentenceTransformers to generate fixed 384 length sentence embedding from feidegger, a zalandoresearch dataset. Then those feature vectors are stored in [RDS for PostgreSQL](https://aws.amazon.com/rds/postgresql/) using extension `pgvector` for product similarity search.

# What is `pgvector`?

pgvector is an open-source extension designed to augment PostgreSQL databases with the capability to store and conduct searches on ML-generated embeddings to identify both exact and approximate nearest neighbors. It’s designed to work seamlessly with other PostgreSQL features, including indexing and querying. 

To generate vector embeddings, you can use ML service such as [Amazon SageMaker](https://aws.amazon.com/sagemaker/) or [Amazon Bedrock](https://aws.amazon.com/bedrock/) (limited preview). SageMaker allows you to easily train and deploy machine learning models, including models that generate vector embeddings for text data.

By utilizing the pgvector extension, PostgreSQL can effectively perform similarity searches on extensive vector embeddings, providing businesses with a speedy and proficient solution. 

Please review pgvector [documentation](https://github.com/pgvector/pgvector) for additional details.

# Solution

![Architecture](static/architecture.png)

## Security

See [CONTRIBUTING](CONTRIBUTING.md#security-issue-notifications) for more information.

## License

This library is licensed under the MIT-0 License. See the LICENSE file.