An open API service indexing awesome lists of open source software.

awesome-llm-and-aigc

🚀🚀🚀A collection of some awesome public projects about Large Language Model(LLM), Vision Language Model(VLM), Vision Language Action(VLA), AI Generated Content(AIGC), the related Datasets and Applications.
https://github.com/coderonion/awesome-llm-and-aigc

Last synced: about 19 hours ago
JSON representation

Applications

Academic Field

  • binary-husky/gpt_academic - : 为ChatGPT/GLM提供图形交互界面,特别优化论文阅读/润色/写作体验,模块化设计,支持自定义快捷按钮&函数插件,支持Python和C++等项目剖析&自译解功能,PDF/LaTex论文翻译&总结功能,支持并行问询多种LLM模型,支持chatglm2等本地模型。兼容文心一言, moss, llama2, rwkv, claude2, 通义千问, 书生, 讯飞星火等。
  • kaixindelele/ChatPaper - : Use ChatGPT to summarize the arXiv papers. 全流程加速科研,利用chatgpt进行论文总结+润色+审稿+审稿回复。 💥💥💥面向全球,服务万千科研人的ChatPaper免费网页版正式上线:https://chatpaper.org/ 💥💥💥
  • GPTZero - The World's #1 AI Detector with over 1 Million Users. Detect ChatGPT, GPT3, GPT4, Bard, and other AI models.
  • BurhanUlTayyab/GPTZero - : An open-source implementation of GPTZero. GPTZero is an AI model with some mathematical formulation to determine if a particular text fed to it is written by AI or a human being.
  • BurhanUlTayyab/DetectGPT - : An open-source Pytorch implementation of DetectGPT. DetectGPT is an amazing method to determine whether a piece of text is written by large language models (like ChatGPT, GPT3, GPT2, BLOOM etc). However, we couldn't find any open-source implementation of it. Therefore this is the implementation of the paper. "DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature". (arXiv 2023).
  • WangRongsheng/ChatGenTitle - : 🌟 ChatGenTitle:使用百万arXiv论文信息在LLaMA模型上进行微调的论文题目生成模型。
  • nishiwen1214/ChatReviewer - : ChatReviewer: use ChatGPT to review papers; ChatResponse: use ChatGPT to respond to reviewers. 💥💥💥ChatReviewer的第一版网页出来了!!! 直接点击:https://huggingface.co/spaces/ShiwenNi/ChatReviewer
  • Shiling42/web-simulator-by-GPT4 - : Online Interactive Physical Simulation Generated by GPT-4. shilingliang.com/web-simulator-by-GPT4/

Adversarial Attack Field

AI Avatar and Digital Human

  • RealChar - : 🎙️🤖Create, Customize and Talk to your AI Character/Companion in Realtime (All in One Codebase!). Have a natural seamless conversation with AI everywhere (mobile, web and terminal) using LLM OpenAI GPT3.5/4, Anthropic Claude2, Chroma Vector DB, Whisper Speech2Text, ElevenLabs Text2Speech🎙️🤖 RealChar.ai/
  • FaceChain - : FaceChain is a deep-learning toolchain for generating your Digital-Twin. FaceChain is a deep-learning toolchain for generating your Digital-Twin. With a minimum of 1 portrait-photo, you can create a Digital-Twin of your own and start generating personal portraits in different settings (multiple styles now supported!). You may train your Digital-Twin model and generate photos via FaceChain's Python scripts, or via the familiar Gradio interface. FaceChain是一个可以用来打造个人数字形象的深度学习模型工具。用户仅需要提供最低三张照片即可获得独属于自己的个人形象数字替身。FaceChain支持在gradio的界面中使用模型训练和推理能力,也支持资深开发者使用python脚本进行训练推理。
  • VirtualWife - : VirtualWife 是一个虚拟主播项目,目前支持在B站进行直播,用户可以自由更换VRM人物模型,大家可以将他作为一个虚拟主播入门demo,在上面扩展自己喜欢功能。
  • GPT-vup - : GPT-vup Live2D数字人直播。GPT-vup BIliBili | 抖音 | AI | 虚拟主播。
  • ChatVRM - : ChatVRMはブラウザで簡単に3Dキャラクターと会話ができるデモアプリケーションです。
  • SillyTavern - : LLM Frontend for Power Users. sillytavern.app
  • HeyGen - Scale your video production with customizable AI avatars. "微信公众号「DataLearner」《《流浪地球2》的数字生命计划可能快实现了!HeyGen即将发布下一代AI真人视频生成技术,效果逼真到无法几乎分辨!》"。
  • VideoChat - : 实时语音交互数字人,支持端到端语音方案(GLM-4-Voice - THG)和级联方案(ASR-LLM-TTS-THG)。可自定义形象与音色,无须训练,支持音色克隆,首包延迟低至3s。Real-time voice interactive digital human, supporting end-to-end voice solutions (GLM-4-Voice - THG) and cascaded solutions (ASR-LLM-TTS-THG). Customizable appearance and voice, supporting voice cloning, with initial package delay as low as 3s.

Animation Field

  • SaaRaaS-1300/InternLM2_horowag - : 🍿InternLM2_Horowag🍿 🍏专门为 2024 书生·浦语大模型挑战赛 (春季赛) 准备的 Repo🍎收录了赫萝相关的微调模型。

Autonomous Driving Field

  • DriveVLM - "DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Models". (CoRL 2024). "微信公众号「清华大学交叉信息研究院」《DriveVLM:清华MARS Lab合作推出首个部署上车的自动驾驶视觉语言大模型》"。
  • UniAD - : "Planning-oriented Autonomous Driving". (CVPR 2023).
  • TransGPT|致远 - : TransGPT是国内首款开源交通大模型,主要致力于在真实交通行业中发挥实际价值。它能够实现交通情况预测、智能咨询助手、公共交通服务、交通规划设计、交通安全教育、协助管理、交通事故报告和分析、自动驾驶辅助系统等功能。TransGPT作为一个通用常识交通大模型,可以为道路工程、桥梁工程、隧道工程、公路运输、水路运输、城市公共交通运输、交通运输经济、交通运输安全等行业提供通识常识。以此为基础,可以落脚到特定的交通应用场景中。
  • LLMLight - : "LLMLight: Large Language Models as Traffic Signal Control Agents". (arXiv 2024).
  • OpenEMMA - : "OpenEMMA: Open-Source Multimodal Model for End-to-End Autonomous Driving". (arXiv 2024).

Chatbot

Code Assistant

  • GPT Pilot - : The first real AI developer. GPT Pilot doesn't just generate code, it builds apps! GPT Pilot is the core technology for the Pythagora VS Code extension that aims to provide the first real AI developer companion. Not just an autocomplete or a helper for PR messages but rather a real AI developer that can write full features, debug them, talk to you about issues, ask for review, etc.
  • StarCoder - : 💫 StarCoder is a language model (LM) trained on source code and natural language text. Its training data incorporates more that 80 different programming languages as well as text extracted from GitHub issues and commits and from notebooks.
  • CodeGeeX2 - : CodeGeeX2: A More Powerful Multilingual Code Generation Model. codegeex.cn
  • Code Llama - : Inference code for CodeLlama models.

Financial Field

Food Field

  • SmartFlowAI/TheGodOfCookery - : 食神(The God Of Cookery)。本项目名称为“食神”( The God Of Cookery ),灵感来自喜剧大师周星驰主演的著名电影《食神》,旨在通过人工智能技术为用户提供烹饪咨询和食谱推荐,帮助用户更好地学习和实践烹饪技巧,降低烹饪门槛,实现《食神》电影中所讲的“只要用心,人人皆能做食神”。

GUI

  • ChatGPT-Admin-Web - : 带有用户管理和后台管理系统的 ChatGPT WebUI. caw.sku.moe
  • 202252197/ChatGPT_JCM - : OpenAI Manage Web. OpenAI管理界面,聚合了OpenAI的所有接口进行界面操作。
  • wieslawsoltes/ChatGPT - : A ChatGPT C# client for graphical user interface runs on MacOS, Windows, Linux, Android, iOS and Browser. Powered by Avalonia UI framework. wieslawsoltes.github.io/ChatGPT/
  • sigoden/aichat - : GUI for ChatGPT API and any LLM. 川虎 Chat 🐯 Chuanhu Chat. 为ChatGPT/ChatGLM/LLaMA/StableLM/MOSS等多种LLM提供了一个轻快好用的Web图形界。
  • amrrs/chatgpt-clone - : Build Yo'own ChatGPT with OpenAI API & Gradio.
  • llama2-webui - : Run Llama 2 locally with gradio UI on GPU or CPU from anywhere (Linux/Windows/Mac). Supporting Llama-2-7B/13B/70B with 8-bit, 4-bit. Supporting GPU inference (6 GB VRAM) and CPU inference.
  • ricklamers/gpt-code-ui - : An open source implementation of OpenAI's ChatGPT Code interpreter.
  • mckaywrigley/chatbot-ui - :An open source ChatGPT UI. chatbotui.com
  • chieapp/chie - : An extensive desktop app for ChatGPT and other LLMs. chie.app
  • AUTOMATIC1111/stable-diffusion-webui - : Stable Diffusion web UI. A browser interface based on Gradio library for Stable Diffusion.
  • Mikubill/sd-webui-controlnet - : ControlNet for Stable Diffusion WebUI. The WebUI extension for ControlNet and other injection-based SD controls.
  • oobabooga/text-generation-webui - : Text generation web UI. A gradio web UI for running Large Language Models like LLaMA, llama.cpp, GPT-J, Pythia, OPT, and GALACTICA.
  • SolidUI - : AI-generated visualization prototyping and editing platform.
  • AIdea - : AIdea 是一款支持 GPT 以及国产大语言模型通义千问、文心一言等,支持 Stable Diffusion 文生图、图生图、 SDXL1.0、超分辨率、图片上色的全能型 APP。
  • Chainlit - : Build Python LLM apps in minutes ⚡️ Chainlit lets you create ChatGPT-like UIs on top of any Python code in minutes! docs.chainlit.io
  • HiveChat - : An AI chat bot for small and medium-sized teams, supporting models such as Deepseek, Open AI, Claude, and Gemini. 专为中小团队设计的 AI 聊天应用,支持 Deepseek、Open AI、Claude、Gemini 等模型。
  • ChatGPT-Next-Web - : A well-designed cross-platform ChatGPT UI (Web / PWA / Linux / Win / MacOS). 一键拥有你自己的跨平台 ChatGPT 应用。
  • cLangUI - : AUI for your AI. Open Source Tailwind components tailored for your GPT, generative AI, and LLM projects.

Humor Generation

IDE

  • Cursor - : An editor made for programming with AI 🤖. Long term, our plan is to build Cursor into the world's most productive development environment. cursor.so

Legal Field

  • ChatLaw - : ChatLaw-法律大模型。chatlaw.cloud/lawchat/
  • LaWGPT - : 🎉 Repo for LaWGPT, Chinese-Llama tuned with Chinese Legal knowledge. LaWGPT 是一系列基于中文法律知识的开源大语言模型。该系列模型在通用中文基座模型(如 Chinese-LLaMA、ChatGLM 等)的基础上扩充法律领域专有词表、大规模中文法律语料预训练,增强了大模型在法律领域的基础语义理解能力。在此基础上,构造法律领域对话问答数据集、中国司法考试数据集进行指令精调,提升了模型对法律内容的理解和执行能力。

Local knowledge Base

  • Langchain-Chatchat - : lLangchain-Chatchat (formerly langchain-ChatGLM), local knowledge based LLM (like ChatGLM) QA app with langchain | 基于 Langchain 与 ChatGLM 等语言模型的本地知识库问答。
  • yanqiangmiffy/Chinese-LangChain - : Chinese-LangChain:中文langchain项目,基于ChatGLM-6b+langchain实现本地化知识库检索与智能答案生成。俗称:小必应,Q.Talk,强聊,QiangTalk。
  • labring/FastGPT - : FastGPT is a knowledge-based question answering system built on the LLM. It offers out-of-the-box data processing and model invocation capabilities. Moreover, it allows for workflow orchestration through Flow visualization, thereby enabling complex question and answer scenarios! fastgpt.run
  • privateGPT - : Ask questions to your documents without an internet connection, using the power of LLMs. 100% private, no data leaves your execution environment at any point. You can ingest documents and ask questions without an internet connection! Built with LangChain, GPT4All, LlamaCpp, Chroma and SentenceTransformers.

Long-Term Memory

Math Field

Medical Field

  • 本草[原名:华驼(HuaTuo)] - : Repo for BenTsao [original name: HuaTuo (华驼)], Llama-7B tuned with Chinese medical knowledge. 本草[原名:华驼(HuaTuo)]: 基于中文医学知识的LLaMA微调模型。本项目开源了经过中文医学指令精调/指令微调(Instruct-tuning) 的LLaMA-7B模型。我们通过医学知识图谱和GPT3.5 API构建了中文医学指令数据集,并在此基础上对LLaMA进行了指令微调,提高了LLaMA在医疗领域的问答效果。 "HuaTuo: Tuning LLaMA Model with Chinese Medical Knowledge". (arXiv 2023).
  • MedSAM - : "Segment Anything in Medical Images". (arXiv 2023). "微信公众号「江大白」《MedSAM在医学领域,图像分割中的落地应用(附论文及源码)》"。
  • LLaVA-Med - : "LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day". (arXiv 2023). "微信公众号「CVHub」《微软发布医学多模态大模型LLaVA-Med | 基于LLaVA的医学指令微调》"。
  • MedicalGPT - : MedicalGPT: Training Your Own Medical GPT Model with ChatGPT Training Pipeline. 训练医疗大模型,实现包括二次预训练、有监督微调、奖励建模、强化学习训练。"微信公众号「KBQA沉思录」《【中文医疗大模型】训练全流程源码剖析》"。
  • MedQA-ChatGLM - : 🛰️ 基于真实医疗对话数据在ChatGLM上进行LoRA、P-Tuning V2、Freeze、RLHF等微调,我们的眼光不止于医疗问答。www.wangrs.co/MedQA-ChatGLM/. "MedQA-ChatGLM: A Medical QA Model Fine-tuned on ChatGLM Using Multiple fine-tuning Method and Real Medical QA Data".
  • xhu248/AutoSAM - : "How to Efficiently Adapt Large Segmentation Model(SAM) to Medical Images". (arXiv 2023).
  • 仲景 - : 仲景:首个实现从预训练到 RLHF 全流程训练的中文医疗大模型。 "Zhongjing: Enhancing the Chinese Medical Capabilities of Large Language Model through Expert Feedback and Real-world Multi-turn Dialogue". (arXiv 2023).
  • DoctorGPT - : DoctorGPT is an LLM that can pass the US Medical Licensing Exam. It works offline, it's cross-platform, & your health data stays private.

Mental Health Field

  • MeChat - : 中文心理健康支持对话数据集(SmileChat)与大模型(MeChat)。 "SMILE: Single-turn to Multi-turn Inclusive Language Expansion via ChatGPT for Mental Health Support". (arXiv 2023).
  • SmartFlowAI/EmoLLM - : EmoLLM-心理健康大模型是一系列能够支持 理解用户-支持用户-帮助用户 心理健康辅导链路的心理健康大模型,由 LLM指令微调而来。心理健康大模型、LLM、The Big Model of Mental Health、Finetune、InternLM2、Qwen、ChatGLM、Baichuan、DeepSeek、Mixtral。

Multi-Agent Collaboration

  • MetaGPT - : "MetaGPT: Meta Programming for Multi-Agent Collaborative Framework". (arXiv 2023).
  • OpenManus - : No fortress, purely open ground. OpenManus is Coming.

Music Field

Object Detection Field

  • DINO - : "DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection". (ICLR 2023).
  • DINOv2 - : "DINOv2: Learning Robust Visual Features without Supervision". (arXiv 2023).
  • GroundingDINO - : "Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection". (ECCV 2024).
  • YOLO-World - : "YOLO-World: Real-Time Open-Vocabulary Object Detection". (CVPR 2024). www.yoloworld.cc
  • Autodistill - : Images to inference with no labeling (use foundation models to train supervised models). Autodistill uses big, slower foundation models to train small, faster supervised models. Using autodistill, you can go from unlabeled images to inference on a custom model running at the edge with no human intervention in between. docs.autodistill.com
  • DOSOD - : "A Light-Weight Framework for Open-Set Object Detection with Decoupled Feature Alignment in Joint Space". (arXiv 2024).

PPT Field

  • SmartSchoolAI/ai-to-pptx - : Ai-to-pptx是一个使用AI技术(DeepSeek)制作PPTX的助手,支持在线生成和导出PPTX。 前端后端同时开源。 主要功能: 1 使用DeepSeek等大语言模型来生成大纲 2 生成PPTX的时候可以选择不同的模板 3 支持导出PPTX。pptx.dandian.net

Question Answering System

  • THUDM/WebGLM - : WebGLM: An Efficient Web-enhanced Question Answering System (KDD 2023). "WebGLM: Towards An Efficient Web-Enhanced Question Answering System with Human Preferences". (arXiv 2023).
  • afaqueumer/DocQA - : Question Answering with Custom FIles using LLM. DocQA 🤖 is a web application built using Streamlit 🔥 and the LangChain 🦜🔗 framework, allowing users to leverage the power of LLMs for Generative Question Answering. 🌟
  • rese1f/MovieChat - : 🔥 chat with over 10K frames of video! MovieChat can handle videos with >10K frames on a 24GB graphics card. MovieChat has a 10000× advantage over other methods in terms of the average increase in GPU memory cost per frame (21.3KB/f to ~200MB/f).

Robotics and Embodied AI

  • LeRobot - : 🤗 LeRobot: State-of-the-art Machine Learning for Real-World Robotics in Pytorch.
  • BestAnHongjun/InternDog - : InternDog: 基于InternLM2大模型的离线具身智能导盲犬。

Role Play

  • KMnO4-zx/xlab-huanhuan - : Chat-甄嬛是利用《甄嬛传》剧本中所有关于甄嬛的台词和语句,基于InternLM2进行LoRA微调或全量微调得到的模仿甄嬛语气的聊天语言模型。
  • JimmyMa99/Roleplay-with-XiYou - : Roleplay-with-XiYou 西游角色扮演。基于《西游记》原文、白话文、ChatGPT生成数据制作的,以InternLM2微调的角色扮演多LLM聊天室。 本项目将介绍关于角色扮演类 LLM 的一切,从数据获取、数据处理,到使用 XTuner 微调并部署至 OpenXLab,再到使用 LMDeploy 部署,以 openai api 的方式接入简单的聊天室,并可以观看不同角色的 LLM 互相交流、互怼。

Speech and Audio Field

Tool Learning

Blogs