{"id":25510373,"url":"https://github.com/first-coding/multimodal-assistant","last_synced_at":"2025-10-30T19:35:15.585Z","repository":{"id":277784198,"uuid":"932757728","full_name":"first-coding/Multimodal-Assistant","owner":"first-coding","description":null,"archived":false,"fork":false,"pushed_at":"2025-02-16T04:12:56.000Z","size":1352,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-05-20T09:09:06.206Z","etag":null,"topics":["ai","deep-learning","faiss","multimodal-deep-learning","python","torch"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/first-coding.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-02-14T13:17:36.000Z","updated_at":"2025-02-16T04:12:59.000Z","dependencies_parsed_at":"2025-02-16T05:17:57.830Z","dependency_job_id":"0aefa1b5-7658-4f96-a7be-42ee0bd053c1","html_url":"https://github.com/first-coding/Multimodal-Assistant","commit_stats":null,"previous_names":["first-coding/multimodal-assistant"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/first-coding/Multimodal-Assistant","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/first-coding%2FMultimodal-Assistant","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/first-coding%2FMultimodal-Assistant/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/first-coding%2FMultimodal-Assistant/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/first-coding%2FMultimodal-Assistant/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/first-coding","download_url":"https://codeload.github.com/first-coding/Multimodal-Assistant/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/first-coding%2FMultimodal-Assistant/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278728292,"owners_count":26035412,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-07T02:00:06.786Z","response_time":59,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","deep-learning","faiss","multimodal-deep-learning","python","torch"],"created_at":"2025-02-19T09:39:19.675Z","updated_at":"2025-10-07T06:00:08.085Z","avatar_url":"https://github.com/first-coding.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"### Multimodal Assistant\n#\n\n### Project Overview\nMultimodal Assistant is an intelligent assistant based on multimodal (text and image) capabilities, integrating features such as image-text retrieval, image captioning, question answering, and sentiment analysis. This project leverages state-of-the-art pre-trained models like CLIP and BLIP to facilitate efficient interaction and understanding between images and text.\n\n![alt text](./data/image.png)\n\n### Features\n- Image-Text Matching: Given a textual description, find the most relevant image (based on the CLIP model).\n\n- Image Captioning: Generate natural language descriptions from images using the BLIP model.\n\n- Sentiment Analysis: Perform sentiment analysis on input text (positive, negative, neutral) using the DistilBERT model.\n\n- Image Question Answering: Given an image and a user's question, the system generates answers related to the image.\n\n### Key Technologies\n- CLIP (Contrastive Language-Image Pretraining): Used for matching similarity between images and text.\n- BLIP (Bootstrapping Image-Language Pretraining): Used for generating natural language descriptions from images and answering image-related questions.\n- DistilBERT: Utilized for sentiment analysis tasks to determine the sentiment of a given text.\n- FAISS (Facebook AI Similarity Search): Used for efficient image vector indexing and retrieval. FAISS enables fast similarity search, allowing the project to perform image retrieval on large-scale image datasets.\n\n### Suggestions and Issues\nI hope this project is helpful to everyone，If you have any suggestions or issues, feel free to discuss them with me through the issues section. Thank you!","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffirst-coding%2Fmultimodal-assistant","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffirst-coding%2Fmultimodal-assistant","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffirst-coding%2Fmultimodal-assistant/lists"}