Projects in Awesome Lists tagged with language-model

https://github.com/huggingface/transformers

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

bert deep-learning flax hacktoberfest jax language-model language-models machine-learning model-hub natural-language-processing nlp nlp-library pretrained-models python pytorch pytorch-transformers seq2seq speech-recognition tensorflow transformer

Last synced: 12 May 2025

https://github.com/microsoft/generative-ai-for-beginners

21 Lessons, Get Started Building with Generative AI 🔗 https://microsoft.github.io/generative-ai-for-beginners/

ai azure chatgpt dall-e generative-ai generativeai gpt language-model llms microsoft-for-beginners openai prompt-engineering semantic-search transformers

Last synced: 12 May 2025

https://microsoft.github.io/generative-ai-for-beginners/

21 Lessons, Get Started Building with Generative AI 🔗 https://microsoft.github.io/generative-ai-for-beginners/

ai azure chatgpt dall-e generative-ai generativeai gpt language-model llms microsoft-for-beginners openai prompt-engineering semantic-search transformers

Last synced: 29 Mar 2025

https://github.com/microsoft/generative-ai-for-beginners?WT.mc_id=academic-122979-leestott

21 Lessons, Get Started Building with Generative AI 🔗 https://microsoft.github.io/generative-ai-for-beginners/

ai azure chatgpt dall-e generative-ai generativeai gpt language-model llms microsoft-for-beginners openai prompt-engineering semantic-search transformers

Last synced: 24 Mar 2025

https://github.com/xtekky/gpt4free

The official gpt4free repository | various collection of powerful language models | o4, o3 and deepseek r1, gpt-4.1, gemini 2.5

chatbot chatbots chatgpt chatgpt-4 chatgpt-api chatgpt-free chatgpt4 deepseek deepseek-api deepseek-r1 gpt gpt-4 gpt-4o gpt4 gpt4-api language-model openai openai-api openai-chatgpt reverse-engineering

Last synced: 12 May 2025

https://github.com/dair-ai/prompt-engineering-guide

🐙 Guides, papers, lecture, notebooks and resources for prompt engineering

chatgpt deep-learning generative-ai language-model openai prompt-engineering

Last synced: 12 May 2025

https://github.com/hiyouga/llama-factory

Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

agent ai chatglm fine-tuning gpt instruction-tuning language-model large-language-models llama llama3 llm lora mistral moe peft qlora quantization qwen rlhf transformers

Last synced: 12 May 2025

https://github.com/rasbt/llms-from-scratch

Implement a ChatGPT-like LLM in PyTorch from scratch, step by step

ai artificial-intelligence chatgpt gpt language-model large-language-models llm python pytorch transformer

Last synced: 12 May 2025

https://github.com/dair-ai/Prompt-Engineering-Guide

🐙 Guides, papers, lecture, notebooks and resources for prompt engineering

chatgpt deep-learning language-model openai prompt-engineering

Last synced: 14 Mar 2025

https://github.com/hiyouga/LLaMA-Factory

Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

agent ai chatglm fine-tuning gpt instruction-tuning language-model large-language-models llama llama3 llm lora mistral moe peft qlora quantization qwen rlhf transformers

Last synced: 14 Mar 2025

https://github.com/laion-ai/open-assistant

OpenAssistant is a chat-based assistant that understands tasks, can interact with third-party systems, and retrieve information dynamically to do so.

ai assistant chatgpt discord-bot language-model machine-learning nextjs python rlhf

Last synced: 14 May 2025

https://github.com/LAION-AI/Open-Assistant

OpenAssistant is a chat-based assistant that understands tasks, can interact with third-party systems, and retrieve information dynamically to do so.

ai assistant chatgpt discord-bot language-model machine-learning nextjs python rlhf

Last synced: 15 Mar 2025

https://github.com/tatsu-lab/stanford_alpaca

Code and documentation to train Stanford's Alpaca models, and generate the data.

deep-learning instruction-following language-model

Last synced: 12 May 2025

https://github.com/deepset-ai/haystack

AI orchestration framework to build customizable, production-ready LLM applications. Connect components (models, vector DBs, file converters) to pipelines or agents that can interact with your data. With advanced retrieval methods, it's best suited for building RAG, question answering, semantic search or conversational agent chatbots.

agents ai bert chatgpt generative-ai information-retrieval language-model large-language-models llm machine-learning nlp python pytorch question-answering rag retrieval-augmented-generation semantic-search squad summarization transformers

Last synced: 12 May 2025

https://github.com/mlc-ai/mlc-llm

Universal LLM Deployment Engine with ML Compilation

language-model llm machine-learning-compilation tvm

Last synced: 13 May 2025

https://github.com/arc53/docsgpt

DocsGPT is an open-source genAI tool that helps users get reliable answers from knowledge source, while avoiding hallucinations. It enables private and reliable information retrieval, with tooling and agentic system capability built in.

ai chatgpt docsgpt hacktoberfest information-retrieval language-model llm machine-learning natural-language-processing python pytorch rag react semantic-search transformers web-app

Last synced: 12 May 2025

https://github.com/yamadashy/repomix

📦 Repomix (formerly Repopack) is a powerful tool that packs your entire repository into a single, AI-friendly file. Perfect for when you need to feed your codebase to Large Language Models (LLMs) or other AI tools like Claude, ChatGPT, DeepSeek, Perplexity, Gemini, Gemma, Llama, Grok, and more.

ai anthropic artificial-intelligence chatbot chatgpt claude deepseek developer-tools gemini genai generative-ai gpt javascript language-model llama llm mcp nodejs openai typescript

Last synced: 13 May 2025

https://github.com/arc53/DocsGPT

DocsGPT is an open-source genAI tool that helps users get reliable answers from knowledge source, while avoiding hallucinations. It enables private and reliable information retrieval, with tooling and agentic system capability built in.

ai chatgpt docsgpt hacktoberfest information-retrieval language-model llm machine-learning natural-language-processing python pytorch rag react semantic-search transformers web-app

Last synced: 14 Mar 2025

https://github.com/mlc-ai/web-llm

High-performance In-browser LLM Inference Engine

chatgpt deep-learning language-model llm tvm webgpu webml

Last synced: 12 May 2025

https://github.com/vercel/ai

The AI Toolkit for TypeScript. From the creators of Next.js, the AI SDK is a free open-source library for building AI-powered applications and agents

anthropic artificial-intelligence gemini generative-ai generative-ui javascript language-model llm nextjs openai react svelte typescript vercel vue

Last synced: 12 May 2025

https://github.com/blinkdl/rwkv-lm

RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.

attention-mechanism chatgpt deep-learning gpt gpt-2 gpt-3 language-model linear-attention lstm pytorch rnn rwkv transformer transformers

Last synced: 09 May 2025

https://github.com/BlinkDL/RWKV-LM

RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.

attention-mechanism chatgpt deep-learning gpt gpt-2 gpt-3 language-model linear-attention lstm pytorch rnn rwkv transformer transformers

Last synced: 20 Mar 2025

https://github.com/vercel-labs/ai

Build AI-powered applications with React, Svelte, Vue, and Solid

artificial-intelligence generative-ai generative-ui huggingface javascript language-model llm nextjs openai react solidjs svelte typescript vercel vue

Last synced: 20 Feb 2025

https://github.com/microsoft/lora

Code for loralib, an implementation of "LoRA: Low-Rank Adaptation of Large Language Models"

adaptation deberta deep-learning gpt-2 gpt-3 language-model lora low-rank pytorch roberta

Last synced: 12 May 2025

https://github.com/mlfoundations/open_clip

An open source implementation of CLIP.

computer-vision contrastive-loss deep-learning language-model multi-modal-learning pretrained-models pytorch zero-shot-classification

Last synced: 12 May 2025

https://github.com/microsoft/LoRA

Code for loralib, an implementation of "LoRA: Low-Rank Adaptation of Large Language Models"

adaptation deberta deep-learning gpt-2 gpt-3 language-model lora low-rank pytorch roberta

Last synced: 27 Mar 2025

https://github.com/neuml/txtai

💡 All-in-one open-source AI framework for semantic search, LLM orchestration and language model workflows

ai artificial-intelligence embeddings information-retrieval language-model large-language-models llm machine-learning nlp python rag retrieval-augmented-generation search search-engine semantic-search sentence-embeddings transformers txtai vector-database vector-search

Last synced: 12 May 2025

https://github.com/speechbrain/speechbrain

A PyTorch-based Speech Toolkit

asr audio audio-processing deep-learning huggingface language-model pytorch speaker-diarization speaker-recognition speaker-verification speech-enhancement speech-processing speech-recognition speech-separation speech-to-text speech-toolkit speechrecognition spoken-language-understanding transformers voice-recognition

Last synced: 13 May 2025

https://github.com/brightmart/nlp_chinese_corpus

大规模中文自然语言处理语料 Large Scale Chinese Corpus for NLP

bert chinese chinese-corpus chinese-dataset chinese-nlp corpus dataset language-model news nlp pretrain question-answering text-classification wiki word2vec

Last synced: 24 Mar 2025

https://github.com/huggingface/tokenizers

💥 Fast State-of-the-Art Tokenizers optimized for Research and Production

bert gpt language-model natural-language-processing natural-language-understanding nlp transformers

Last synced: 26 Mar 2025

https://github.com/blinkdl/chatrwkv

ChatRWKV is like ChatGPT but powered by RWKV (100% RNN) language model, and open source.

chatbot chatgpt language-model pytorch rnn rwkv

Last synced: 13 May 2025

https://github.com/BlinkDL/ChatRWKV

ChatRWKV is like ChatGPT but powered by RWKV (100% RNN) language model, and open source.

chatbot chatgpt language-model pytorch rnn rwkv

Last synced: 15 Mar 2025

https://neuml.github.io/txtai/

💡 All-in-one open-source embeddings database for semantic search, LLM orchestration and language model workflows

embeddings information-retrieval language-model large-language-models llm machine-learning neural-search nlp python rag retrieval-augmented-generation search search-engine semantic-search sentence-embeddings transformers txtai vector-database vector-search vector-search-engine

Last synced: 16 Jan 2025

https://github.com/eleutherai/lm-evaluation-harness

A framework for few-shot evaluation of language models.

evaluation-framework language-model transformer

Last synced: 14 May 2025

https://github.com/optimalscale/lmflow

An Extensible Toolkit for Finetuning and Inference of Large Foundation Models. Large Models for All.

chatgpt deep-learning instruction-following language-model pretrained-models pytorch transformer

Last synced: 14 May 2025

https://github.com/EleutherAI/gpt-neo

An implementation of model parallel GPT-2 and GPT-3-style models using the mesh-tensorflow library.

gpt gpt-2 gpt-3 language-model transformers

Last synced: 02 Apr 2025

https://github.com/OptimalScale/LMFlow

An Extensible Toolkit for Finetuning and Inference of Large Foundation Models. Large Models for All.

chatgpt deep-learning instruction-following language-model pretrained-models pytorch transformer

Last synced: 28 Mar 2025

https://optimalscale.github.io/LMFlow/

An Extensible Toolkit for Finetuning and Inference of Large Foundation Models. Large Models for All.

chatgpt deep-learning instruction-following language-model pretrained-models pytorch transformer

Last synced: 16 Jan 2025

https://github.com/eleutherai/gpt-neox

An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries

deepspeed-library gpt-3 language-model transformers

Last synced: 12 May 2025

https://github.com/EleutherAI/lm-evaluation-harness

A framework for few-shot evaluation of language models.

evaluation-framework language-model transformer

Last synced: 23 Mar 2025

https://github.com/EleutherAI/gpt-neox

An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries

deepspeed-library gpt-3 language-model transformers

Last synced: 27 Mar 2025

https://github.com/opennmt/opennmt-py

Open Source Neural Machine Translation and (Large) Language Models in PyTorch

deep-learning language-model llms machine-translation neural-machine-translation pytorch

Last synced: 13 May 2025

https://github.com/LostRuins/koboldcpp

Run GGUF models easily with a KoboldAI UI. One File. Zero Install.

gemma ggml gguf koboldai koboldcpp language-model llama llamacpp llm mistral

Last synced: 23 Mar 2025

https://github.com/OpenNMT/OpenNMT-py

Open Source Neural Machine Translation and (Large) Language Models in PyTorch

deep-learning language-model llms machine-translation neural-machine-translation pytorch

Last synced: 30 Mar 2025

https://github.com/thudm/cogvlm

a state-of-the-art-level open visual language model | 多模态预训练模型

cross-modality language-model multi-modal pretrained-models visual-language-models

Last synced: 14 May 2025

https://github.com/THUDM/CogVLM

a state-of-the-art-level open visual language model | 多模态预训练模型

cross-modality language-model multi-modal pretrained-models visual-language-models

Last synced: 28 Mar 2025

https://github.com/codertimo/bert-pytorch

Google AI 2018 BERT pytorch implementation

bert language-model nlp pytorch transformer

Last synced: 13 May 2025

https://github.com/codertimo/BERT-pytorch

Google AI 2018 BERT pytorch implementation

bert language-model nlp pytorch transformer

Last synced: 02 Apr 2025

https://github.com/aiwaves-cn/agents

An Open-source Framework for Data-centric, Self-evolving Autonomous Language Agents

autonomous-agents language-model llm

Last synced: 13 May 2025

https://github.com/nexaai/nexa-sdk

Nexa SDK is a comprehensive toolkit for supporting GGML and ONNX models. It supports text generation, image generation, vision-language models (VLM), Audio Language Model, auto-speech-recognition (ASR), and text-to-speech (TTS) capabilities.

asr audio edge-computing language-model llm on-device-ai on-device-ml sdk sdk-python stable-diffusion transformers tts vlm whisper

Last synced: 11 May 2025

https://github.com/yizhongw/self-instruct

Aligning pretrained language models with instruction data generated by themselves.

general-purpose-model instruction-tuning language-model

Last synced: 13 Apr 2025

https://github.com/NexaAI/nexa-sdk

Nexa SDK is a comprehensive toolkit for supporting GGML and ONNX models. It supports text generation, image generation, vision-language models (VLM), Audio Language Model, auto-speech-recognition (ASR), and text-to-speech (TTS) capabilities.

asr audio edge-computing language-model llm on-device-ai on-device-ml sdk sdk-python stable-diffusion transformers tts vlm whisper

Last synced: 07 Feb 2025

https://github.com/cluebenchmark/clue

中文语言理解测评基准 Chinese Language Understanding Evaluation Benchmark: datasets, baselines, pre-trained models, corpus and leaderboard

albert benchmark bert chinese chineseglue corpus dataset glue language-model nlu pretrained-models pytorch roberta tensorflow transformers

Last synced: 14 May 2025

https://github.com/CLUEbenchmark/CLUE

中文语言理解测评基准 Chinese Language Understanding Evaluation Benchmark: datasets, baselines, pre-trained models, corpus and leaderboard

albert benchmark bert chinese chineseglue corpus dataset glue language-model nlu pretrained-models pytorch roberta tensorflow transformers

Last synced: 28 Mar 2025

https://github.com/microsoft/lmops

General technology for enabling AI capabilities w/ LLMs and MLLMs

agi gpt language-model llm lm lmops nlp pretraining prompt promptist x-prompt

Last synced: 13 May 2025

https://github.com/xlang-ai/OpenAgents

[COLM 2024] OpenAgents: An Open Platform for Language Agents in the Wild

agent assistant-chat-bots code-generation executable-langauge-grounding gpt hacktoberfest language-model language-model-agent llm semantic-parsing tool-learning ui

Last synced: 28 Mar 2025

https://github.com/xlang-ai/openagents

[COLM 2024] OpenAgents: An Open Platform for Language Agents in the Wild

agent assistant-chat-bots code-generation executable-langauge-grounding gpt hacktoberfest language-model language-model-agent llm semantic-parsing tool-learning ui

Last synced: 14 May 2025

https://github.com/eth-sri/lmql

A language for constraint-guided and efficient LLM programming.

chatgpt huggingface language-model programming-language

Last synced: 14 May 2025

https://github.com/microsoft/LMOps

General technology for enabling AI capabilities w/ LLMs and MLLMs

agi gpt language-model llm lm lmops nlp pretraining prompt promptist x-prompt

Last synced: 13 Mar 2025

https://github.com/mlfoundations/open_flamingo

An open-source framework for training large multimodal models.

computer-vision deep-learning flamingo in-context-learning language-model multimodal-learning pytorch

Last synced: 09 Apr 2025

https://github.com/ravenscroftj/turbopilot

Turbopilot is an open source large-language-model based code completion engine that runs locally on CPU

code-completion cpp language-model machine-learning

Last synced: 17 Jan 2025

https://github.com/hiyouga/chatglm-efficient-tuning

Fine-tuning ChatGLM-6B with PEFT | 基于 PEFT 的高效 ChatGLM 微调

alpaca chatglm chatglm2 chatgpt fine-tuning huggingface language-model lora peft pytorch qlora rlhf transformers

Last synced: 19 Jan 2025

https://github.com/hiyouga/ChatGLM-Efficient-Tuning

Fine-tuning ChatGLM-6B with PEFT | 基于 PEFT 的高效 ChatGLM 微调

alpaca chatglm chatglm2 chatgpt fine-tuning huggingface language-model lora peft pytorch qlora rlhf transformers

Last synced: 29 Mar 2025

https://github.com/huggingface/speech-to-speech

Speech To Speech: an effort for an open-sourced and modular GPT4-o

ai assistant language-model machine-learning python speech speech-synthesis speech-to-text speech-translation

Last synced: 31 Dec 2024

https://github.com/xtekky/chatgpt-clone

ChatGPT interface with better UI

chatgpt chatgpt-api chatgpt-app chatgpt-clone gpt-4 gpt-4-api gpt-interface gpt3 gpt4 gpt4-api gpt4all interface language language-model site ui

Last synced: 14 May 2025

https://github.com/langroid/langroid

Harness LLMs with Multi-Agent Programming

agents ai chatgpt function-calling gpt gpt-4 gpt4 information-retrieval language-model llama llm llm-agent llm-framework local-llm multi-agent-systems openai-api rag retrieval-augmented-generation

Last synced: 12 May 2025

https://github.com/docta-ai/docta

A Doctor for your data

data data-centric-ai data-centric-machine-learning data-curation data-diagnosis language-model rlhf

Last synced: 13 May 2025

https://github.com/getzep/zep

Zep | The Memory Foundation For Your AI Stack

ai langchain language-model llamaindex llm vectorsearch

Last synced: 03 Apr 2025

https://github.com/microsoft/phicookbook

This is a Phi Family of SLMs book for getting started with Phi Models. Phi a family of open sourced AI models developed by Microsoft. Phi models are the most capable and cost-effective small language models (SLMs) available, outperforming models of the same size and next size up across a variety of language, reasoning, coding, and math benchmarks

cookbook language-model phi-4 phi-4-mini phi-4-multimodal phi3 phi3-testing phi3-vision phi4 phi4-mini phi4-multimodal slm small-language-model

Last synced: 14 May 2025

https://github.com/cvi-szu/linly

Chinese-LLaMA 1&2、Chinese-Falcon 基础模型；ChatFlow中文对话模型；中文OpenLLaMA模型；NLP预训练/指令微调数据集

bert chatbot chatgpt chinese chinese-nlp gpt-3 language-model llama nlp zero-shot-learning

Last synced: 15 May 2025

https://github.com/CVI-SZU/Linly

Chinese-LLaMA 1&2、Chinese-Falcon 基础模型；ChatFlow中文对话模型；中文OpenLLaMA模型；NLP预训练/指令微调数据集

bert chatbot chatgpt chinese chinese-nlp gpt-3 language-model llama nlp zero-shot-learning

Last synced: 13 Mar 2025

https://github.com/salesforce/codet5

Home of CodeT5: Open Code LLMs for Code Understanding and Generation

code-generation code-intelligence code-understanding language-model large-language-models

Last synced: 15 May 2025

https://github.com/salesforce/CodeT5

Home of CodeT5: Open Code LLMs for Code Understanding and Generation

code-generation code-intelligence code-understanding language-model large-language-models

Last synced: 16 Mar 2025

https://github.com/swe-bench/swe-bench

SWE-bench [Multimodal]: Can Language Models Resolve Real-world Github Issues?

benchmark language-model software-engineering

Last synced: 12 May 2025

https://langroid.github.io/langroid/

Harness LLMs with Multi-Agent Programming

agents ai chatgpt function-calling gpt gpt-4 gpt4 information-retrieval language-model llama llm llm-agent llm-framework local-llm multi-agent-systems openai-api rag retrieval-augmented-generation

Last synced: 16 Jan 2025

https://github.com/tensorflow/lingvo

Lingvo

asr distributed gpu-computing language-model lm machine-translation mnist nlp research seq2seq speech speech-recognition speech-synthesis speech-to-text tensorflow translation tts

Last synced: 13 May 2025

https://github.com/Docta-ai/docta

A Doctor for your data

data data-centric-ai data-centric-machine-learning data-curation data-diagnosis language-model rlhf

Last synced: 26 Mar 2025

https://github.com/internlm/internlm-xcomposer

InternLM-XComposer2.5-OmniLive: A Comprehensive Multimodal System for Long-term Streaming Video and Audio Interactions

chatgpt foundation gpt gpt-4 instruction-tuning language-model large-language-model large-vision-language-model llm mllm multi-modality multimodal supervised-finetuning vision-language-model vision-transformer visual-language-learning

Last synced: 15 May 2025

https://github.com/InternLM/InternLM-XComposer

InternLM-XComposer2.5-OmniLive: A Comprehensive Multimodal System for Long-term Streaming Video and Audio Interactions

chatgpt foundation gpt gpt-4 instruction-tuning language-model large-language-model large-vision-language-model llm mllm multi-modality multimodal supervised-finetuning vision-language-model vision-transformer visual-language-learning

Last synced: 07 May 2025

https://github.com/stochasticai/xturing

Build, customize and control you own LLMs. From data pre-processing to fine-tuning, xTuring provides an easy way to personalize open-source LLMs. Join our discord community: https://discord.gg/TgHXuSJEk6

adapter alpaca deep-learning fine-tuning finetuning gen-ai generative-ai gpt-2 gpt-j language-model llama llm lora mistral mixed-precision peft quantization

Last synced: 15 May 2025

https://github.com/stochasticai/xTuring

Build, customize and control you own LLMs. From data pre-processing to fine-tuning, xTuring provides an easy way to personalize open-source LLMs. Join our discord community: https://discord.gg/TgHXuSJEk6

adapter alpaca deep-learning fine-tuning finetuning gen-ai generative-ai gpt-2 gpt-j language-model llama llm lora mistral mixed-precision peft quantization