Projects in Awesome Lists tagged with language-models

https://github.com/huggingface/transformers

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

bert deep-learning flax hacktoberfest jax language-model language-models machine-learning model-hub natural-language-processing nlp nlp-library pretrained-models python pytorch pytorch-transformers seq2seq speech-recognition tensorflow transformer

Last synced: 16 Dec 2024

https://github.com/bigscience-workshop/petals

🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading

bloom chatbot deep-learning distributed-systems falcon gpt guanaco language-models large-language-models llama machine-learning mixtral neural-networks nlp pipeline-parallelism pretrained-models pytorch tensor-parallelism transformer volunteer-computing

Last synced: 16 Dec 2024

https://github.com/argosopentech/argos-translate

Open-source offline translation library written in Python

language-models linux machine-translation nlp open-source python transformers translation

Last synced: 16 Dec 2024

https://github.com/jalammar/ecco

Explain, analyze, and visualize NLP language models. Ecco creates interactive visualizations directly in Jupyter notebooks explaining the behavior of Transformer-based language models (like GPT2, BERT, RoBERTA, T5, and T0).

explorables language-models natural-language-processing nlp pytorch visualization

Last synced: 17 Dec 2024

https://github.com/deepset-ai/farm

:house_with_garden: Fast & easy transfer learning for NLP. Harvesting language models for the industry. Focus on Question Answering.

bert deep-learning germanbert language-models ner nlp nlp-framework nlp-library pretrained-models pytorch question-answering roberta transfer-learning xlnet-pytorch

Last synced: 19 Dec 2024

https://github.com/deepset-ai/FARM

:house_with_garden: Fast & easy transfer learning for NLP. Harvesting language models for the industry. Focus on Question Answering.

bert deep-learning germanbert language-models ner nlp nlp-framework nlp-library pretrained-models pytorch question-answering roberta transfer-learning xlnet-pytorch

Last synced: 04 Nov 2024

https://github.com/curiousily/get-things-done-with-prompt-engineering-and-langchain

LangChain & Prompt Engineering tutorials on Large Language Models (LLMs) such as ChatGPT with custom data. Jupyter notebooks on loading and indexing data, creating prompt templates, CSV agents, and using retrieval QA chains to query the custom data. Projects for using a private LLM (Llama 2) for chat with PDF files, tweets sentiment analysis.

artificial-intelligence chatgpt deep-learning gpt-4 gpt4 langchain language-models large-language-models llama2 openai prompt-engineering python

Last synced: 15 Dec 2024

https://github.com/curiousily/Get-Things-Done-with-Prompt-Engineering-and-LangChain

LangChain & Prompt Engineering tutorials on Large Language Models (LLMs) such as ChatGPT with custom data. Jupyter notebooks on loading and indexing data, creating prompt templates, CSV agents, and using retrieval QA chains to query the custom data. Projects for using a private LLM (Llama 2) for chat with PDF files, tweets sentiment analysis.

artificial-intelligence chatgpt deep-learning gpt-4 gpt4 langchain language-models large-language-models llama2 openai prompt-engineering python

Last synced: 24 Oct 2024

https://github.com/declare-lab/tango

A family of diffusion models for text-to-audio generation.

audio-generation diffusion diffusion-models language-models large-language-models text-to-audio

Last synced: 20 Dec 2024

https://github.com/kennethleungty/llama-2-open-source-llm-cpu-inference

Running Llama 2 and other Open-Source LLMs on CPU Inference Locally for Document Q&A

c-transformers chatgpt cpu cpu-inference deep-learning document-qa faiss langchain language-models large-language-models llama llama-2 llm machine-learning natural-language-processing nlp open-source-llm python sentence-transformers transformers

Last synced: 15 Dec 2024

https://github.com/kennethleungty/Llama-2-Open-Source-LLM-CPU-Inference

Running Llama 2 and other Open-Source LLMs on CPU Inference Locally for Document Q&A

c-transformers chatgpt cpu cpu-inference deep-learning document-qa faiss langchain language-models large-language-models llama llama-2 llm machine-learning natural-language-processing nlp open-source-llm python sentence-transformers transformers

Last synced: 16 Nov 2024

https://github.com/zjunlp/prompt4reasoningpapers

[ACL 2023] Reasoning with Language Model Prompting: A Survey

arithmetic-reasoning artificial-intelligence awsome-list chain-of-thought chatgpt commonsense-reasoning datasets gpt-3 language-models large-language-models llm logical-reasoning natural-language-processing nlp paper-list prompt prompt-engineering reasoning survey symbolic-reasoning

Last synced: 09 Nov 2024

https://github.com/zjunlp/Prompt4ReasoningPapers

[ACL 2023] Reasoning with Language Model Prompting: A Survey

arithmetic-reasoning artificial-intelligence awsome-list chain-of-thought chatgpt commonsense-reasoning datasets gpt-3 language-models large-language-models llm logical-reasoning natural-language-processing nlp paper-list prompt prompt-engineering reasoning survey symbolic-reasoning

Last synced: 24 Oct 2024

https://github.com/princeton-nlp/lm-bff

[ACL 2021] LM-BFF: Better Few-shot Fine-tuning of Language Models https://arxiv.org/abs/2012.15723

few-shot-learning language-models lm-bff

Last synced: 21 Dec 2024

https://github.com/princeton-nlp/LM-BFF

[ACL 2021] LM-BFF: Better Few-shot Fine-tuning of Language Models https://arxiv.org/abs/2012.15723

few-shot-learning language-models lm-bff

Last synced: 06 Nov 2024

https://github.com/VinAIResearch/PhoBERT

PhoBERT: Pre-trained language models for Vietnamese (EMNLP-2020 Findings)

bert bert-embeddings deep-learning fairseq language-models named-entity-recognition natural-language-inference ner nli part-of-speech-tagging phobert pos-tagging python3 rdrsegmenter roberta transformers transformers-library vietnamese vietnamese-nlp vncorenlp

Last synced: 04 Nov 2024

https://github.com/hazyresearch/hyena-dna

Official implementation for HyenaDNA, a long-range genomic foundation model built with Hyena

foundation-models genomics language-models

Last synced: 15 Dec 2024

https://github.com/webis-de/small-text

Active Learning for Text Classification in Python

active-learning deep-learning language-models looking-for-contributors machine-learning natural-language-processing nlp python pytorch small-language-models text-classification transformers

Last synced: 19 Dec 2024

https://github.com/monarch-initiative/ontogpt

LLM-based ontological extraction tools, including SPIRES

ai chat-gpt data-modeling gpt-3 information-extraction language-models large-language-models linkml llm monarchinitiative named-entity-recognition ner nlp oaklib obofoundry relation-extraction

Last synced: 04 Nov 2024

https://github.com/bigscience-workshop/xmtf

Crosslingual Generalization through Multitask Finetuning

bloom bloomz instruction-tuning language-models large-language-models mt0 multilingual-nlp multitask-learning t5 zero-shot-learning

Last synced: 14 Dec 2024

https://github.com/EricFillion/happy-transformer

Happy Transformer makes it easy to fine-tune and perform inference with NLP Transformer models.

ai artificial-intelligence bert deep-learning language-models machine-learning natural-language-processing nlp python question-answering roberta text-classification transformers

Last synced: 03 Sep 2024

https://github.com/cedrickchee/chatgpt-universe

ChatGPT Universe is fleeting notes on ChatGPT, GPT, and large language models (LLMs)

chatgpt generative-model gpt language-models resource-list

Last synced: 16 Dec 2024

https://github.com/neurocult/agency

🕵️‍♂️ Library designed for developers eager to explore the potential of Large Language Models (LLMs) and other generative AI through a clean, effective, and Go-idiomatic approach.

agents ai artificial-general-intelligence artificial-intelligence artificial-neural-networks autonomous-agents chatgpt generative-ai go golang gpt language-models llm llmops machine-learning neural-network nlp openai rag vector-database

Last synced: 06 Nov 2024

https://github.com/cli99/llm-analysis

Latency and Memory Analysis of Transformer Models for Training and Inference

analysis deep-learning language-model language-models machine-learning nlp transformers

Last synced: 24 Nov 2024

https://github.com/huggingface/datablations

Scaling Data-Constrained Language Models

gpt high-performance-computing language-models large-language-models llms scaling-laws

Last synced: 29 Nov 2024

https://github.com/petals-infra/chat.petals.dev

💬 Chatbot web app + HTTP and Websocket endpoints for LLM inference with the Petals client

api bloom chatbot distributed-systems gpt guanaco language-models large-language-models llama llama2 transformer volunteer-computing

Last synced: 15 Dec 2024

https://github.com/extreme-bert/extreme-bert

ExtremeBERT is a toolkit that accelerates the pretraining of customized language models on customized datasets, described in the paper “ExtremeBERT: A Toolkit for Accelerating Pretraining of Customized BERT”.

bert deep-learning language-model language-models machine-learning natural-language-processing nlp python pytorch transformer

Last synced: 16 Nov 2024

https://github.com/agencyenterprise/PromptInject

PromptInject is a framework that assembles prompts in a modular fashion to provide a quantitative analysis of the robustness of LLMs to adversarial prompt attacks. 🏆 Best Paper Awards @ NeurIPS ML Safety Workshop 2022

adversarial-attacks agi agi-alignment ai-alignment ai-safety chain-of-thought gpt-3 language-models large-language-models machine-learning ml-safety prompt-engineering

Last synced: 31 Oct 2024

https://github.com/agencyenterprise/promptinject

PromptInject is a framework that assembles prompts in a modular fashion to provide a quantitative analysis of the robustness of LLMs to adversarial prompt attacks. 🏆 Best Paper Awards @ NeurIPS ML Safety Workshop 2022

adversarial-attacks agi agi-alignment ai-alignment ai-safety chain-of-thought gpt-3 language-models large-language-models machine-learning ml-safety prompt-engineering

Last synced: 15 Dec 2024

https://github.com/neulab/knn-transformers

PyTorch + HuggingFace code for RetoMaton: "Neuro-Symbolic Language Modeling with Automaton-augmented Retrieval" (ICML 2022), including an implementation of kNN-LM and kNN-MT

huggingface knn knn-lm knn-mt knn-transformers knnlm knnmt language language-models machine models nearest nearest-neighbor neighbor neuro-symbolic pytorch retomaton transformers translation

Last synced: 18 Dec 2024

https://github.com/bhattbhavesh91/voice-assistant-whisper-chatgpt

This repository will guide you to create your own Smart Virtual Assistant like Google Assistant using Open AI's ChatGPT, Whisper. The entire solution is created using Python & Gradio.

chatgpt chatgpt-api google-assistant gpt-3 gradio huggingface language-model language-models openapi virtual-assistant voice-assistant whisper

Last synced: 17 Dec 2024

https://github.com/epfl-dlab/aiflows

🤖🌊 aiFlows: The building blocks of your collaborative AI

agent agents ai ai-framework ai-frameworks chatgpt copilot gpt language-model language-models llm llms open-source oss python

Last synced: 06 Nov 2024

https://github.com/sea-snell/jaxseq

Train very large language models in Jax.

deep-learning flax gpt2 gpt3 huggingface jax language-models opt

Last synced: 06 Dec 2024

https://github.com/picovoice/picollm

On-device LLM Inference Powered by X-Bit Quantization

compression efficient-inference gemma generative-ai language-model language-models large-language-model llama llama2 llama3 llm llm-inference llms mistral mixtral model-compression natural-language-processing quantization self-hosted

Last synced: 15 Dec 2024

https://github.com/tomekkorbak/pretraining-with-human-feedback

Code accompanying the paper Pretraining Language Models with Human Preferences

ai-alignment ai-safety decision-transformers gpt language-models pretraining reinforcement-learning rlhf

Last synced: 19 Dec 2024

https://github.com/quanta-quest/quanta-quest

AI-powered universal search for all your personal data, tailored just for you. Goal:The world's first product with "edge-side LLMs + consumer data localization" as its core development direction.

agent ai anthropic bert chatgpt claude edge-computing gpt huggingface knowledgebase language-models llm nextjs nlp personal-ass rag semantic-vector-search transformers universal-search workflow

Last synced: 17 Dec 2024

https://github.com/bilel-bj/ROSGPT_Vision

Commanding robots using only Language Models' prompts

chatgpt language-models language-models-are-next large-language-models llm prompt-engineering prompting-robotic-modalities robotic-design-patterns robotic-vision robotics ros2 visual-language-models

Last synced: 29 Oct 2024

https://github.com/dmitryryumin/emnlp-2023-papers

EMNLP 2023 Papers: Explore cutting-edge research from EMNLP 2023, the premier conference for advancing empirical methods in natural language processing. Stay updated on the latest in machine learning, deep learning, and natural language processing with code included. :star: support NLP!

bert computational-linguistics emnlp emnlp2023 gpt language-models llms machine-learning machine-translation multilingual-nlp named-entity-recognition natural-language-processing ner nlp nlp-applications sentiment-analysis syntax-and-semantics text-mining transformers word-embeddings

Last synced: 15 Nov 2024

https://github.com/Loodos/turkish-language-models

Transformer based Turkish language models

language-models natural-language-processing nlp turkish

Last synced: 12 Nov 2024

https://github.com/naver/disco

A Toolkit for Distributional Control of Generative Models

ai alignment distributional-policy-gradients fine-tuning generative-models human-preferences language-models machine-learning monte-carlo-sampling

Last synced: 08 Nov 2024

https://github.com/nicolay-r/AREkit

Document level Attitude and Relation Extraction toolkit (AREkit) for sampling and processing large text collections with ML and for ML

bert datasets frames language-models neural-networks nlp pandas pandas-dataframe prompt prompting relation-extraction sentiment-analysis tensorflow

Last synced: 01 Nov 2024

https://github.com/anyks/alm

Smart Language Model

alm arpa cpp language-models tokenization tokenizer vocab-pruning

Last synced: 11 Nov 2024

https://github.com/pbloem/language-models

Keras implementations of three language models: character-level RNN, word-level RNN and Sentence VAE (Bowman, Vilnis et al 2016).

bowman keras language-models rnn-language-model vae

Last synced: 14 Nov 2024

https://github.com/retarfi/language-pretraining

Pre-training Language Models for Japanese

bert electra implementation japanese language-model language-models natural-language-processing nlp pytorch transformer transformers

Last synced: 15 Nov 2024

https://github.com/bhattbhavesh91/diffusion-chatgpt

This repository will guide you to create your Images via Stable Diffusion using a Smart Virtual Assistant like Google Assistant using Open AI's ChatGPT, Whisper. The entire solution is created using Python & Gradio.

chatgpt chatgpt-api google-assistant gpt-3 gradio gradio-interface language-model language-models openai stable-diffusion stable-diffusion-diffusers stable-diffusion-v2 whisper

Last synced: 16 Nov 2024

https://github.com/alan-turing-institute/robots-in-disguise

Information and materials for the Turing's "robots-in-disguise" reading group on fundamental AI research.

deep-learning diffusion-models foundation-model hut23 language-models large-language-models machine-learning nlp transformers

Last synced: 19 Dec 2024

https://github.com/alexandra-chron/ntua-slp-wassa-iest2018

Deep-learning Transfer Learning models of NTUA-SLP team submitted at the IEST of WASSA 2018 at EMNLP 2018.

deep-learning deep-neural-networks emotion-analysis language-models lstm python pytorch sentiment-analysis transfer-learning twitter

Last synced: 05 Nov 2024

https://github.com/adrianbzg/llama-multimodal-vqa

Multimodal Instruction Tuning for Llama 3

chatbot chatgpt gpt-4 huggingface instruction-tuning language-models llama llama2 llama3 multimodal multimodal-instruction-tuning visual-language-learning visual-question-answering vqa

Last synced: 10 Oct 2024

https://github.com/cmungall/semantic-llama

A knowledge extraction tool that uses a large language model to extract semantic information from text

ai knowledge-extraction language-models linkml oaklib obofoundry

Last synced: 22 Oct 2024

https://github.com/lucidrains/nim-tokenizer

Implementation of a simple BPE tokenizer, but in Nim

artificial-intelligence deep-learning language-models nim tokenizer

Last synced: 10 Dec 2024

https://github.com/yueyuel/reliablelm4code

Collections of research, benchmarks and tools towards more robust and reliable language models for code; LM4Code; LM4SE; reliable LLM; LLM4Code

code-generation code-intelligence language-models llm4code lm4se reliability software-

Last synced: 11 Nov 2024

https://github.com/nicolay-r/rusentrel-leaderboard

This is an official Leaderboard for the RuSentRel-1.1 dataset originally described in paper (arxiv:1808.08932)

attention attention-mechanism benchmark bert-model bilstm chatgpt classifiers cnn language-models leaderboard low-resource-nlp neural-networks relation-extraction sentiment-analysis

Last synced: 19 Dec 2024

https://github.com/dillondaudert/pssp_lstm

Recurrent neural network implementations for protein secondary structure prediction and language models

amino-acid-sequence deep-learning deep-neural-networks jupyter-notebook language-models lstm paper prediction pretrained-models protein python3 recurrent-neural-networks rnn secondary structure structure-prediction tensorflow unsupervised-learning

Last synced: 12 Oct 2024

https://github.com/vgherard/kgrams

k-grams, Language Models, and All That

language-models n-grams natural-language-processing

Last synced: 13 Dec 2024

https://github.com/skblaz/attviz

Dissecting Transformers via attention visualization

attention-is-all-you-need attention-mechanism interactive language-model language-models machine-learning node-js nodejs python visualization visualizations

Last synced: 17 Dec 2024

https://github.com/pro-genai/auto-trendy-keywords

Real-time AI-driven Trending keyword generation for SEO

ai artificial-intelligence arxiv gen-ai genai generative-ai generativeai language-models large-language-models llm llms prompt-engineering python research research-paper research-project seo seo-friendly seo-optimization seo-tools

Last synced: 22 Nov 2024

https://github.com/medoidai/givebackgpt

An early version of a system that credits creators based on the similarity of their content to an LLM response. Giving back to creators is the only way for fair, sustainable AI economies that lead to true growth.

ai-ethics bootstrap chatbot css embeddings generative-ai html intellectual-property javascript language-models open-source responsive-web-design sustainable-ai web-search

Last synced: 13 Nov 2024

https://github.com/centre-for-humanities-computing/danish-ner-bias

Investigating bias in Danish language models in Named Entity Recognition (NER). Code from the paper titled "Detecting intersectionality in NER models: A data-driven approach."

language-models named-entity-recognition nlp

Last synced: 09 Nov 2024

https://github.com/joel-beck/readnext

Hybrid Recommender System for Computer Science Papers | Master's Thesis Project 2023

citation-analysis hybrid-recommender-system language-models python recommender-system

Last synced: 05 Nov 2024

https://github.com/vaasudevans/natural-language-processing-assignments

UNB Fall-2018 NLP Assignments 💬

baseline bigrams hidden-markov-model information-retrieval-based-chatbot language-models nlp python27 sentiment-analysis unb unigram

Last synced: 13 Dec 2024

https://github.com/linhaowei1/molretrieval

This repo implements many methods to retrieve molecules that are similar to a target molecule from a large molecule corpus.

ai4science biology computational-biology language-models molecule rag retreival retrieval-augmented-generation

Last synced: 09 Oct 2024

https://github.com/temilaj/nlp-coronavirus-wiki-twitter-perplexity

Natural language processing project to visualize word choice patterns from coronavirus (and related) articles, and compute the average perplexity scores of language models generated from these articles when used with tweets about the subject matter

coronavirus covid-19 language-models n-grams natural-language-processing nlp perplexity-scores

Last synced: 12 Nov 2024

https://github.com/tomekkorbak/kl-gpt3

A modular library for evaluating KL between a Huggingface Transformers models and GPT3

gpt3 language-models

Last synced: 17 Dec 2024

https://github.com/divanvisagie/ratatoskr-prototype

Experiments with ChatGPT, Notion and telegram

ai chatgpt language-models llm

Last synced: 13 Dec 2024

https://github.com/lukexyz/language-models

:earth_africa::book::speech_balloon: Sentiment analysis and text generation using BERT and ULMFiT (2018)

bert language-models transformer ulm-fit

Last synced: 21 Dec 2024

https://github.com/yash-kavaiya/30-days-llm-mastery-course

30-Days-LLM-Mastery-Course: A comprehensive, hands-on course diving deep into Large Language Models (LLMs). From foundational concepts to advanced techniques, learn to build, train, and deploy state-of-the-art language models.

attention-mechanism fine-tuning language-models llm model-deployment nlp pytorch transformers

Last synced: 09 Nov 2024

https://github.com/eric11eca/saint-nli

A new evaluation mechanism and a learning strategy for de-biased and interpretable NLI models. Models co-learn sentence classification and evidence retrieval for the classification.

computational-semantics language-models natural-language-inference transformers

Last synced: 21 Nov 2024

https://github.com/infinitode/duplipy

DupliPy is a quick and easy-to-use package that can handle text formatting and data augmentation tasks for NLP in Python. It now offers support for image augmentation tasks as well.

ai augmentation data-analysis data-preprocessing data-science images language-models nlp preprocessing text-data text-datasets text-formatting

Last synced: 08 Nov 2024

https://github.com/raul23/simple-transformer-tts

This project offers a deeper exploration of tttzof351's "Simple Transformer TTS" codebase, enhanced with insights from Gemini, Google AI's advanced language model.

educational language-models pytorch text-to-speech transformer-models

Last synced: 14 Nov 2024

https://github.com/nicolay-r/bert-utils-for-attitude-extraction

Data Utils for BERT models in Sentiment Attitude Extraction task

bert language-models relation-extraction sentiment-analysis

Last synced: 19 Dec 2024

https://github.com/quanta-quest/quanta-quest-app

AI-powered universal search for all your personal data, tailored just for you. Goal:The world's first product with "edge-side LLMs + consumer data localization" as its core development direction.

agent ai bert claude edge-computing gpt huggingface knowledgebase language-models nextjs nlp rag transformers wails workflow

Last synced: 20 Nov 2024

https://github.com/terilios/automated_data_scientist

Automated Data Scientist: An intelligent, adaptive data analysis tool that leverages AI-driven automation to dynamically plan, execute, and refine data science workflows. Automatically handles data preparation, analysis planning, code generation, and result interpretation using advanced language models.

adaptive-analytics ai-driven-analytics ai-powered-data-tools api-integration automated-data-science automation data-insights data-preparation data-science-workflow data-visualization dynamic-analysis-planning exploratory-data-analysis intelligent-data-processing language-models machine-learning ml-ops openai-gpt python scalable-data-analysis

Last synced: 11 Nov 2024