{"id":28403000,"url":"https://github.com/rbbrdckybk/minigpt-4","last_synced_at":"2025-06-26T17:31:32.210Z","repository":{"id":160133140,"uuid":"635055381","full_name":"rbbrdckybk/MiniGPT-4","owner":"rbbrdckybk","description":"Simplified local Windows OS setup of MiniGPT-4 running in an Anaconda environment; includes example local server and client.","archived":false,"fork":false,"pushed_at":"2023-12-27T23:44:31.000Z","size":113,"stargazers_count":9,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-06-02T01:16:05.184Z","etag":null,"topics":["artificial-intelligence","chatgpt","computer-vision","machine-learning","minigpt4","vicuna"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rbbrdckybk.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null},"funding":{"github":"rbbrdckybk","patreon":null,"open_collective":null,"ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"otechie":null,"lfx_crowdfunding":null,"custom":null}},"created_at":"2023-05-01T21:40:04.000Z","updated_at":"2024-07-20T22:44:18.000Z","dependencies_parsed_at":"2023-12-28T01:02:25.284Z","dependency_job_id":"601a105e-612b-4e5a-a726-a791bf6ed8e2","html_url":"https://github.com/rbbrdckybk/MiniGPT-4","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/rbbrdckybk/MiniGPT-4","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rbbrdckybk%2FMiniGPT-4","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rbbrdckybk%2FMiniGPT-4/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rbbrdckybk%2FMiniGPT-4/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rbbrdckybk%2FMiniGPT-4/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rbbrdckybk","download_url":"https://codeload.github.com/rbbrdckybk/MiniGPT-4/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rbbrdckybk%2FMiniGPT-4/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262113224,"owners_count":23260983,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["artificial-intelligence","chatgpt","computer-vision","machine-learning","minigpt4","vicuna"],"created_at":"2025-06-01T16:36:21.020Z","updated_at":"2025-06-26T17:31:32.189Z","avatar_url":"https://github.com/rbbrdckybk.png","language":"Python","funding_links":["https://github.com/sponsors/rbbrdckybk"],"categories":[],"sub_categories":[],"readme":"# MiniGPT-4\n\nSimplified local setup of [MiniGPT-4](https://github.com/Vision-CAIR/MiniGPT-4) running in an Anaconda environment. Fixes for various Windows OS issues are provided, as well as links to pre-prepared Vicuna weights.\n\nI've also included a simple MiniGPT-4 server that you can run locally that will respond to API requests, along with an example client that demonstrates how to interact with it.\n\n# Requirements\n\nYou'll need an Nvidia GPU with at least 12GB of VRAM (24GB+ is preferred). These instructions were tested on a Windows 10 machine with an Nvidia 3080Ti GPU, but should work on Linux as well (not tested).\n\n# Setup\n\n**[1]** Install [Anaconda](https://www.anaconda.com/products/individual), open the root terminal, and create a new environment (and activate it):\n```\nconda create --name minigpt4 python=3.9\nconda activate minigpt4\n```\n\n**[2]** Install a couple required Python packages:\n```\nconda install -c anaconda git urllib3\n```\n\n**[3]** Clone the official MiniGPT-4 repository and switch to its directory:\n```\ngit clone https://github.com/Vision-CAIR/MiniGPT-4.git\ncd MiniGPT-4\n```\n\n**[4]** Install requirements:\n```\ncurl -L -o requirements.txt -C - \"https://raw.githubusercontent.com/rbbrdckybk/MiniGPT-4/main/requirements.txt\"\npip install -r requirements.txt\n```\nNote that if you get an error while installing pycocotools on Windows, you may need to install the [Microsoft C++ Build Tools](https://visualstudio.microsoft.com/visual-cpp-build-tools/). See [this issue](https://github.com/cocodataset/cocoapi/issues/169#issuecomment-724622726) for more information.\n\n**[5]** Install PyTorch:\n```\nconda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia\n```\n\n**[6]** Prepare your own weights by following the official instructions (step 2) [here](https://github.com/Vision-CAIR/MiniGPT-4#installation). This involves waiting for your access request to be approved, then downloading ~200GB of LLaMA weights, and then using the Vicuna toolset to prepare working weights.\n\n**-OR-** \n\nSimply reference these pre-prepared weights (credit to [wangrongsheng](https://huggingface.co/wangrongsheng)) by opening **minigpt4/configs/models/minigpt4.yaml** \u0026 edit line 16:\n\nif you have 24GB+ of VRAM:\n```\nllama_model: \"wangrongsheng/MiniGPT-4-LLaMA\"\n```\nif you have 12GB+ of VRAM:\n```\nllama_model: \"wangrongsheng/MiniGPT-4-LLaMA-7B\"\n```\n\n**[7]** Download and reference the pretrained MiniGPT-4 checkpoint (links provided by official repo here @ [step 3](https://github.com/Vision-CAIR/MiniGPT-4#installation)):\n\nif you have 24GB+ of VRAM, [download this checkpoint](https://drive.google.com/file/d/1a4zLvaiDBr-36pasffmgpvH5P7CKmpze/view?usp=share_link).\n\nif you have 12GB+ of VRAM, [download this checkpoint](https://drive.google.com/file/d/1RY9jV0dyqLX-o38LrumkKRh6Jtaop58R/view?usp=sharing).\n\nPlace the downloaded checkpoint file in your MiniGPT-4 directory, then open **eval_configs/minigpt4_eval.yaml** and modify line 11:\n\nif you have 24GB+ of VRAM:\n```\nckpt: 'pretrained_minigpt4.pth'\n```\nif you have 12GB+ of VRAM (feel free to correct the typo in the official filename):\n```\nckpt: 'prerained_minigpt4_7b.pth'\n```\n\n**[8]** If you're on Windows, you'll need to run these commands to [fix a known issue with bitsandbytes](https://github.com/TimDettmers/bitsandbytes/issues/175):\n```\npip uninstall bitsandbytes\npip install git+https://github.com/Keith-Hon/bitsandbytes-windows.git\n```\nYou'll also need to place [this DLL](https://github.com/DeXtmL/bitsandbytes-win-prebuilt/blob/main/libbitsandbytes_cuda116.dll) into your **[Anaconda root directory]\\envs\\textgen\\lib\\site-packages\\bitsandbytes** folder.\n\nSkip this step entirely if you're on Linux!\n\n**[9]** (optional) Download my simple API server \u0026 client implementation: I've removed gradio and set MiniGPT-4 up as a simple Flask server that you can run locally to handle API requests. I've also coded a simple client example so you can see how to interact with it.\n```\npip install Flask\ncurl -L -o api-server.py -C - \"https://raw.githubusercontent.com/rbbrdckybk/MiniGPT-4/main/api-server.py\"\ncurl -L -o api-client-example.py -C - \"https://raw.githubusercontent.com/rbbrdckybk/MiniGPT-4/main/api-client-example.py\"\nmkdir img\ncurl -L -o img/simpsons.jpg -C - \"https://raw.githubusercontent.com/rbbrdckybk/MiniGPT-4/main/img/simpsons.jpg\"\n```\nSee below for usage instructions!\n\n# Usage\n\nRun the official gradio demo to verify that everything works:\n```\npython demo.py --cfg-path eval_configs/minigpt4_eval.yaml --gpu-id 0\n```\nNote that several large files (~15GB total) will be downloaded on the first run.\n\nIf you downloaded my API server \u0026 client in step 9 (verify that the official gradio demo works properly before continuing!), you can test them by starting the server with:\n```\npython api-server.py\n```\nOnce the server is running, you can start the client with:\n```\npython api-client-example.py\n```\nYou should see the client send the example image (img/simpsons.jpg) to the server and ask MiniGPT-4 several questions about it. Import the **MiniGPT4_Client** class from api-client-example.py into your own projects to easily interact with MiniGPT-4!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frbbrdckybk%2Fminigpt-4","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frbbrdckybk%2Fminigpt-4","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frbbrdckybk%2Fminigpt-4/lists"}