{"id":23431076,"url":"https://github.com/activeloopai/deeplake-bedrock","last_synced_at":"2025-04-09T15:10:59.262Z","repository":{"id":265810023,"uuid":"886265823","full_name":"activeloopai/deeplake-bedrock","owner":"activeloopai","description":null,"archived":false,"fork":false,"pushed_at":"2024-11-10T16:27:28.000Z","size":960,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-09T15:10:55.135Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/activeloopai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-11-10T16:16:26.000Z","updated_at":"2024-11-10T16:27:32.000Z","dependencies_parsed_at":"2024-12-01T02:51:51.498Z","dependency_job_id":"95e9e83d-0cb6-4fb2-924e-a291a91f0064","html_url":"https://github.com/activeloopai/deeplake-bedrock","commit_stats":null,"previous_names":["activeloopai/deeplake-bedrock"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/activeloopai%2Fdeeplake-bedrock","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/activeloopai%2Fdeeplake-bedrock/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/activeloopai%2Fdeeplake-bedrock/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/activeloopai%2Fdeeplake-bedrock/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/activeloopai","download_url":"https://codeload.github.com/activeloopai/deeplake-bedrock/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248055276,"owners_count":21040157,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-23T09:49:28.149Z","updated_at":"2025-04-09T15:10:59.236Z","avatar_url":"https://github.com/activeloopai.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n# Multimodal Retrieval with Deep Lake, Bedrock, and AI Integrations \nThis project showcases a multimodal retrieval system combining the **Deep Lake**  REST API, **Bedrock** , and **OpenAI**  for image and text processing. It allows users to query a dataset with text prompts, retrieve relevant images, and leverage AI models for context-aware responses.\n## Overview \n\nThis solution is designed to enable multimodal data retrieval and answer generation by:\n \n- **Querying Datasets** : Supports querying datasets in Deep Lake and retrieving top matches based on text-based queries.\n \n- **AI-Powered Insights** : Uses Bedrock and OpenAI models for answering questions about retrieved images or text segments.\n \n- **Deep Memory** : Integrates deep memory retrieval to enhance context-aware responses, leveraging the Deep Lake database for improved AI-driven answers.\n\n## Project Components \n1. Main Retrieval and Processing (`main.py`)\n\n    The core script:\n\n    - Queries Deep Lake datasets based on user-defined prompts.\n\n    - Uses Bedrock and Deep Lake integrations to retrieve relevant images.\n    \n    - Leverages functions from `bedrock_code.py` and `deeplake_deepmemory.py` to generate context-aware answers from images and text.\n\n    - Saves and displays retrieved images for each query.\n2. Bedrock Integration (`bedrock_code.py`)\n\n    This file provides:\n    \n    - Functions to send messages to Bedrock’s AI models (e.g., Claude 3 Sonnet) using the AWS `boto3` client.\n\n    - Supports both image and text-based question-answering by converting inputs into compatible formats for Bedrock’s AI models.\n\n    - A sample use case for processing an image with Bedrock’s API.\n3. Deep Memory and Embedding (`deeplake_deepmemory.py`)\n\n    This module includes:\n\n    - Functions to retrieve context from a Deep Lake vector database based on user queries.\n\n    - An embedding function using OpenAI’s embedding model for efficient text similarity and context-aware retrieval.\n\n    - Options to enable or disable deep memory\n\n\n4. Utility Functions (`utils.py`)\n\n    Helper functions to:\n\n    - Convert images to byte format for API compatibility.\n\n    - Structure messages for both text and image prompts, formatting them to interact with Bedrock and OpenAI models.\n\n    - Generate embeddings for text inputs using OpenAI’s API.\n\n5. Notebook Examples (`notebook_bedrock.ipynb` and `notebook.ipynb`)\n\n    Notebooks demonstrate how to use the multimodal retrieval system:\n \n    - **Multimodal Retrieval with Bedrock and Deep Lake** : Examples of setting up queries, sending them to the Deep Lake API, and processing responses.\n    \n    - **Visualization** : Displays images retrieved based on query results and shows how Bedrock can answer questions about these images.\n6. Requirements (`requirements.txt`)\nThe necessary Python libraries for this project:\n \n    - `deeplake V3`: For handling datasets and vector stores.\n    \n    - `openai`: To access embedding functions for text processing.\n\n## Installation \n\nInstall the necessary dependencies:\n\n\n```bash\npip install -r requirements.txt\n```\n\n## Environment Setup \n\nDefine environment variables for secure access to APIs:\n \n- `TOKEN`: Deep Lake API access token.\n \n- `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`: AWS credentials for Bedrock.\n \n- `ACTIVELOOP_TOKEN`: Token for ActiveLoop’s Deep Lake.\n \n- `OPENAI_API_KEY`: OpenAI API key for embedding functions.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Factiveloopai%2Fdeeplake-bedrock","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Factiveloopai%2Fdeeplake-bedrock","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Factiveloopai%2Fdeeplake-bedrock/lists"}