An open API service indexing awesome lists of open source software.

Natural language processing

Natural language processing (NLP) is a field of computer science that studies how computers and humans interact. In the 1950s, Alan Turing published an article that proposed a measure of intelligence, now called the Turing test. More modern techniques, such as deep learning, have produced results in the fields of language modeling, parsing, and natural-language tasks.

https://github.com/huggingface/transformers

🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

audio deep-learning deepseek gemma glm hacktoberfest llm machine-learning model-hub natural-language-processing nlp pretrained-models python pytorch pytorch-transformers qwen speech-recognition transformer vlm

Last synced: 21 Nov 2026

https://github.com/666ghj/bettafish

微舆:人人可用的多Agent舆情分析助手,打破信息茧房,还原舆情原貌,预测未来走向,辅助决策!从0实现,不依赖任何框架。

agent-framework data-analysis deep-research deep-search llms multi-agent-system nlp public-opinion-analysis python3 sentiment-analysis

Last synced: 18 Oct 2026

https://github.com/google-research/bert

TensorFlow code and pre-trained models for BERT

google natural-language-processing natural-language-understanding nlp tensorflow

Last synced: 18 Oct 2026

https://github.com/google/langextract

A Python library for extracting structured information from unstructured text using LLMs with precise source grounding and interactive visualization.

gemini gemini-ai gemini-api gemini-flash gemini-pro information-extration large-language-models llm nlp python structured-data

Last synced: 21 Nov 2026

https://github.com/hankcs/hanlp

中文分词 词性标注 命名实体识别 依存句法分析 成分句法分析 语义依存分析 语义角色标注 指代消解 风格转换 语义相似度 新词发现 关键词短语提取 自动摘要 文本分类聚类 拼音简繁转换 自然语言处理

dependency-parser hanlp named-entity-recognition natural-language-processing nlp pos-tagging semantic-parsing text-classification

Last synced: 18 Oct 2026

https://github.com/hankcs/HanLP

中文分词 词性标注 命名实体识别 依存句法分析 成分句法分析 语义依存分析 语义角色标注 指代消解 风格转换 语义相似度 新词发现 关键词短语提取 自动摘要 文本分类聚类 拼音简繁转换 自然语言处理

dependency-parser hanlp named-entity-recognition natural-language-processing nlp pos-tagging semantic-parsing text-classification

Last synced: 18 Mar 2025

https://github.com/nandhakishorm/laya

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

calibration classification decision-model huggingface jev modernbert multilingual nlp python pytorch routing typed-decisions zero-shot

Last synced: 18 Oct 2026

https://github.com/stanford-oval/storm

An LLM-powered knowledge curation system that researches a topic and generates a full-length report with citations.

agentic-rag deep-research emnlp2024 knowledge-curation large-language-models naacl nlp report-generation retrieval-augmented-generation

Last synced: 21 Nov 2026

https://github.com/nirdiamant/rag_techniques

This repository showcases various advanced techniques for Retrieval-Augmented Generation (RAG) systems. Each technique has a detailed notebook tutorial.

agentic-rag ai embeddings generative-ai gpt langchain llama-index llm llms machine-learning nlp openai python rag retrieval-augmented-generation semantic-search tutorials vector-database

Last synced: 21 Nov 2026

https://github.com/accumulatemore/cv

✅(已完结)超级全面的 深度学习 笔记【土堆 Pytorch】【李沐 动手学深度学习】【吴恩达 深度学习】【大飞 大模型Agent】

agent agents book chinese computer-vision cv deep-learning jupyter-notebook llm llms machine-learning natural-language-processing nlp notebook python rag

Last synced: 21 Nov 2026

https://github.com/microsoft/unilm

Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

beit beit-3 bitnet deepnet document-ai foundation-models kosmos kosmos-1 layoutlm layoutxlm llm minilm mllm multimodal nlp pre-trained-model textdiffuser trocr unilm xlm-e

Last synced: 21 Nov 2026

https://github.com/huggingface/datasets

🤗 The largest hub of ready-to-use datasets for AI models with fast, easy-to-use and efficient data manipulation tools

ai artificial-intelligence computer-vision dataset-hub datasets deep-learning huggingface llm machine-learning natural-language-processing nlp numpy pandas pytorch speech tensorflow

Last synced: 21 Nov 2026

https://github.com/ai4finance-foundation/fingpt

FinGPT: Open-Source Financial Large Language Models! Revolutionize 🔥 We release the trained model on HuggingFace.

chatgpt finance fingpt fintech large-language-models machine-learning nlp prompt-engineering pytorch reinforcement-learning robo-advisor sentiment-analysis technical-analysis

Last synced: 21 Nov 2026

https://github.com/rasahq/rasa

💬 Open source machine learning framework to automate text- and voice-based conversations: NLU, dialogue management, connect to Slack, Facebook, and more - Create chatbots and voice assistants

bot bot-framework botkit bots chatbot chatbots chatbots-framework conversation-driven-development conversational-agents conversational-ai conversational-bots machine-learning machine-learning-library mitie natural-language-processing nlp nlu rasa spacy wit

Last synced: 21 Nov 2026

https://github.com/ymcui/chinese-llama-alpaca

中文LLaMA&Alpaca大语言模型+本地CPU/GPU训练部署 (Chinese LLaMA & Alpaca LLMs)

alpaca alpaca-2 large-language-models llama llama-2 llm lora nlp plm pre-trained-language-models quantization

Last synced: 21 Nov 2026

https://github.com/ymcui/Chinese-LLaMA-Alpaca

中文LLaMA&Alpaca大语言模型+本地CPU/GPU训练部署 (Chinese LLaMA & Alpaca LLMs)

alpaca alpaca-2 large-language-models llama llama-2 llm lora nlp plm pre-trained-language-models quantization

Last synced: 13 Mar 2025

https://github.com/RasaHQ/rasa

💬 Open source machine learning framework to automate text- and voice-based conversations: NLU, dialogue management, connect to Slack, Facebook, and more - Create chatbots and voice assistants

bot bot-framework botkit bots chatbot chatbots chatbots-framework conversation-driven-development conversational-agents conversational-ai conversational-bots machine-learning machine-learning-library mitie natural-language-processing nlp nlu rasa spacy wit

Last synced: 14 Mar 2025

https://github.com/nlp-love/ml-nlp

此项目是机器学习(Machine Learning)、深度学习(Deep Learning)、NLP面试中常考到的知识点和代码实现,也是作为一个算法工程师必会的理论基础知识。

deep-learning machine-learning nlp

Last synced: 21 Nov 2026

https://github.com/dair-ai/ml-youtube-courses

📺 Discover the latest machine learning / AI courses on YouTube.

ai data-science deep-learning machine-learning natural-language-processing nlp

Last synced: 21 Nov 2026

https://github.com/memvid/memvid

Memory layer for AI Agents. Replace complex RAG pipelines with a serverless, single-file memory layer. Give your agents instant retrieval and long-term memory.

ai context embedded faiss knowledge-base knowledge-graph llm machine-learning memory memvid mv2 nlp offline-first opencv python rag retrieval-augmented-generation semantic-search vector-database video-processing

Last synced: 21 Nov 2026

https://github.com/unstructured-io/unstructured

Convert documents to structured data effortlessly. Unstructured is open-source ETL solution for transforming complex documents into clean, structured formats for language models. Visit our website to learn more about our enterprise grade Platform product for production grade workflows, partitioning, enrichments, chunking and embedding.

data-pipelines deep-learning document-image-analysis document-image-processing document-parser document-parsing docx donut information-retrieval langchain llm machine-learning ml natural-language-processing nlp ocr pdf pdf-to-json pdf-to-text preprocessing

Last synced: 21 Nov 2026

https://github.com/NLP-LOVE/ML-NLP

此项目是机器学习(Machine Learning)、深度学习(Deep Learning)、NLP面试中常考到的知识点和代码实现,也是作为一个算法工程师必会的理论基础知识。

deep-learning machine-learning nlp

Last synced: 09 Apr 2025

https://github.com/AI4Finance-Foundation/FinGPT

FinGPT: Open-Source Financial Large Language Models! Revolutionize 🔥 We release the trained model on HuggingFace.

chatgpt finance fingpt fintech large-language-models machine-learning nlp prompt-engineering pytorch reinforcement-learning robo-advisor sentiment-analysis technical-analysis

Last synced: 28 Mar 2025

https://github.com/botpress/botpress

The open-source hub to build & deploy GPT/LLM Agents ⚡️

agent ai botpress chatbot chatgpt gpt gpt-4 langchain llm nlp openai prompt

Last synced: 21 Nov 2026

https://github.com/nvidia/deeplearningexamples

State-of-the-Art Deep Learning scripts organized by models - easy to train and deploy with reproducible accuracy and performance on enterprise-grade infrastructure.

computer-vision deep-learning drug-discovery forecasting large-language-models mxnet nlp paddlepaddle pytorch recommender-systems speech-recognition speech-synthesis tensorflow tensorflow2 translation

Last synced: 21 Nov 2026

https://github.com/graykode/nlp-tutorial

Natural Language Processing Tutorial for Deep Learning Researchers

attention bert natural-language-processing nlp paper pytorch tensorflow transformer tutorial

Last synced: 18 Oct 2026

https://github.com/flairnlp/flair

A very simple framework for state-of-the-art Natural Language Processing (NLP)

machine-learning named-entity-recognition natural-language-processing nlp pytorch semantic-role-labeling sequence-labeling word-embeddings

Last synced: 21 Nov 2026

https://github.com/dair-ai/ML-YouTube-Courses

📺 Discover the latest machine learning / AI courses on YouTube.

ai data-science deep-learning machine-learning natural-language-processing nlp

Last synced: 14 Mar 2025

https://github.com/NVIDIA/DeepLearningExamples

State-of-the-Art Deep Learning scripts organized by models - easy to train and deploy with reproducible accuracy and performance on enterprise-grade infrastructure.

computer-vision deep-learning drug-discovery forecasting large-language-models mxnet nlp paddlepaddle pytorch recommender-systems speech-recognition speech-synthesis tensorflow tensorflow2 translation

Last synced: 17 Mar 2025

https://github.com/flairNLP/flair

A very simple framework for state-of-the-art Natural Language Processing (NLP)

machine-learning named-entity-recognition natural-language-processing nlp pytorch semantic-role-labeling sequence-labeling word-embeddings

Last synced: 14 Mar 2025

https://github.com/tangyudi/ai-learn

人工智能学习路线图,整理近200个实战案例与项目,免费提供配套教材,零基础入门,就业实战!包括:Python,数学,机器学习,数据分析,深度学习,计算机视觉,自然语言处理,PyTorch tensorflow machine-learning,deep-learning data-analysis data-mining mathematics data-science artificial-intelligence python tensorflow tensorflow2 caffe keras pytorch algorithm numpy pandas matplotlib seaborn nlp cv等热门领域

algorithm artificial-intelligence caffe cv data-analysis data-mining data-science deep-learning keras machine-learning mathematics matplotlib nlp numpy pandas python pytorch seaborn tensorflow tensorflow2

Last synced: 21 Nov 2026

https://github.com/dair-ai/ai-papers-of-the-week

🔥Highlighting the top ML papers every week.

ai data-science deeplearning machine-learning nlp

Last synced: 21 Nov 2026

https://github.com/dair-ai/AI-Papers-of-the-Week

🔥Highlighting the top ML papers every week.

ai data-science deeplearning machine-learning nlp

Last synced: 16 Nov 2026

https://github.com/PaddlePaddle/PaddleNLP

👑 Easy-to-use and powerful NLP and LLM library with 🤗 Awesome model zoo, supporting wide-range of NLP tasks from research to industrial applications, including 🗂Text Classification, 🔍 Neural Search, ❓ Question Answering, ℹ️ Information Extraction, 📄 Document Intelligence, 💌 Sentiment Analysis etc.

bert compression distributed-training document-intelligence embedding ernie information-extraction llama llm neural-search nlp paddlenlp pretrained-models question-answering search-engine semantic-analysis sentiment-analysis transformers uie

Last synced: 18 Mar 2025

https://github.com/spencermountain/compromise

modest natural-language processing

named-entity-recognition nlp part-of-speech

Last synced: 18 Oct 2026

https://github.com/data-privacy-stack/presidio

An open-source framework for detecting, redacting, masking, and anonymizing sensitive data (PII) across text, images, and structured data. Supports NLP, pattern matching, and customizable pipelines.

anonymization data-anonymization data-masking data-obfuscation data-privacy data-redaction de-identification guardrails image-redactor named-entity-recognition nlp personally-identifiable-information phi pii pii-detection privacy python sensitive-data spacy transformers

Last synced: 21 Nov 2026

https://github.com/allenai/allennlp

An open-source NLP research library, built on PyTorch.

data-science deep-learning natural-language-processing nlp python pytorch

Last synced: 18 Oct 2026

https://github.com/huggingface/tokenizers

💥 Fast State-of-the-Art Tokenizers optimized for Research and Production

bert gpt language-model natural-language-processing natural-language-understanding nlp transformers

Last synced: 21 Nov 2026

https://github.com/huggingface/text-generation-inference

Large Language Model Text Generation Inference

bloom deep-learning falcon gpt inference nlp pytorch starcoder transformer

Last synced: 18 Nov 2026

https://github.com/chiphuyen/stanford-tensorflow-tutorials

This repository contains code examples for the Stanford's course: TensorFlow for Deep Learning Research.

chatbot course-materials deep-learning machine-learning natural-language-processing nlp python stanford tensorflow tutorial

Last synced: 18 Oct 2026

https://github.com/ymcui/chinese-bert-wwm

Pre-Training with Whole Word Masking for Chinese BERT(中文BERT-wwm系列模型)

bert bert-wwm bert-wwm-ext chinese-bert nlp pytorch rbt roberta roberta-wwm tensorflow

Last synced: 18 Oct 2026

https://github.com/mooler0410/llmspracticalguide

A curated list of practical guide resources of LLMs (LLMs Tree, Examples, Papers)

large-language-models natural-language-processing nlp survey

Last synced: 21 Nov 2026

https://github.com/dair-ai/ML-Papers-of-the-Week

🔥Highlighting the top ML papers every week.

ai data-science deeplearning machine-learning nlp

Last synced: 17 Mar 2025

https://github.com/dair-ai/ml-papers-of-the-week

🔥Highlighting the top ML papers every week.

ai data-science deeplearning machine-learning nlp

Last synced: 27 Jan 2026

https://github.com/stanfordnlp/corenlp

CoreNLP: A Java suite of core NLP tools for tokenization, sentence segmentation, NER, parsing, coreference, sentiment analysis, etc.

named-entity-recognition natural-language-processing nlp nlp-parsing stanford-nlp

Last synced: 18 Oct 2026

https://stanfordnlp.github.io/CoreNLP/

CoreNLP: A Java suite of core NLP tools for tokenization, sentence segmentation, NER, parsing, coreference, sentiment analysis, etc.

named-entity-recognition natural-language-processing nlp nlp-parsing stanford-nlp

Last synced: 23 Apr 2025

https://github.com/jadore801120/attention-is-all-you-need-pytorch

A PyTorch implementation of the Transformer model in "Attention is All You Need".

attention attention-is-all-you-need deep-learning natural-language-processing nlp pytorch

Last synced: 21 Nov 2026

https://github.com/Mooler0410/LLMsPracticalGuide

A curated list of practical guide resources of LLMs (LLMs Tree, Examples, Papers)

large-language-models natural-language-processing nlp survey

Last synced: 04 Apr 2025

https://github.com/tangyudi/Ai-Learn

人工智能学习路线图,整理近200个实战案例与项目,免费提供配套教材,零基础入门,就业实战!包括:Python,数学,机器学习,数据分析,深度学习,计算机视觉,自然语言处理,PyTorch tensorflow machine-learning,deep-learning data-analysis data-mining mathematics data-science artificial-intelligence python tensorflow tensorflow2 caffe keras pytorch algorithm numpy pandas matplotlib seaborn nlp cv等热门领域

algorithm artificial-intelligence caffe cv data-analysis data-mining data-science deep-learning keras machine-learning mathematics matplotlib nlp numpy pandas python pytorch seaborn tensorflow tensorflow2

Last synced: 07 May 2025

https://github.com/ymcui/Chinese-BERT-wwm

Pre-Training with Whole Word Masking for Chinese BERT(中文BERT-wwm系列模型)

bert bert-wwm bert-wwm-ext chinese-bert nlp pytorch rbt roberta roberta-wwm tensorflow

Last synced: 28 Mar 2025

https://github.com/sloria/textblob

Simple, Pythonic, text processing--Sentiment analysis, part-of-speech tagging, noun phrase extraction, translation, and more.

natural-language-processing nlp nltk pattern python

Last synced: 18 Oct 2026

https://github.com/stanfordnlp/CoreNLP

CoreNLP: A Java suite of core NLP tools for tokenization, sentence segmentation, NER, parsing, coreference, sentiment analysis, etc.

named-entity-recognition natural-language-processing nlp nlp-parsing stanford-nlp

Last synced: 18 Mar 2025

https://github.com/modelscope/modelscope

ModelScope: bring the notion of Model-as-a-Service to life.

cv deep-learning machine-learning multi-modal nlp python science speech

Last synced: 21 Nov 2026

https://github.com/sloria/TextBlob

Simple, Pythonic, text processing--Sentiment analysis, part-of-speech tagging, noun phrase extraction, translation, and more.

natural-language-processing nlp nltk pattern python python-3

Last synced: 14 Mar 2025

https://github.com/woooodyy/llm-agent-paper-list

The paper list of the 86-page SCIS cover paper "The Rise and Potential of Large Language Model Based Agents: A Survey" by Zhiheng Xi et al.

agent large-language-models llm nlp survey

Last synced: 21 Nov 2026

https://github.com/maartengr/bertopic

Leveraging BERT and c-TF-IDF to create easily interpretable topics.

bert ldavis machine-learning nlp sentence-embeddings topic topic-modeling topic-modelling topic-models transformers

Last synced: 21 Nov 2026

https://github.com/stanfordnlp/stanza

Stanford NLP Python library for tokenization, sentence segmentation, NER, and parsing of many human languages

artificial-intelligence corenlp deep-learning machine-learning named-entity-recognition natural-language-processing nlp python pytorch universal-dependencies

Last synced: 21 Nov 2026

https://github.com/morizeyao/gpt2-chinese

Chinese version of GPT2 training code, using BERT tokenizer.

chinese gpt-2 nlp text-generation transformer

Last synced: 21 Nov 2026

https://github.com/Morizeyao/GPT2-Chinese

Chinese version of GPT2 training code, using BERT tokenizer.

chinese gpt-2 nlp text-generation transformer

Last synced: 24 Mar 2025

https://github.com/WooooDyy/LLM-Agent-Paper-List

The paper list of the 86-page paper "The Rise and Potential of Large Language Model Based Agents: A Survey" by Zhiheng Xi et al.

agent large-language-models llm nlp survey

Last synced: 16 Mar 2025

https://github.com/ymcui/Chinese-LLaMA-Alpaca-2

中文LLaMA-2 & Alpaca-2大模型二期项目 + 64K超长上下文模型 (Chinese LLaMA-2 & Alpaca-2 LLMs with 64K long context models)

64k alpaca alpaca-2 alpaca2 flash-attention large-language-models llama llama-2 llama2 llm nlp rlhf yarn

Last synced: 24 Mar 2025

https://github.com/adbar/trafilatura

Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML

article-extractor corpus-builder corpus-tools crawler html-to-markdown html2text llm news-aggregator news-crawler nlp rag readability rss-feed scraping tei text-cleaning text-extraction text-mining text-preprocessing web-scraping

Last synced: 21 Nov 2026

https://github.com/ymcui/chinese-llama-alpaca-2

中文LLaMA-2 & Alpaca-2大模型二期项目 + 64K超长上下文模型 (Chinese LLaMA-2 & Alpaca-2 LLMs with 64K long context models)

64k alpaca alpaca-2 alpaca2 flash-attention large-language-models llama llama-2 llama2 llm nlp rlhf yarn

Last synced: 21 Nov 2026

https://github.com/thunlp/wantwords

An open-source online reverse dictionary.

natural-language-processing nlp reverse-dictionary word

Last synced: 18 Oct 2026

https://github.com/thunlp/WantWords

An open-source online reverse dictionary.

natural-language-processing nlp reverse-dictionary word

Last synced: 27 Mar 2025

https://github.com/paddlepaddle/models

Officially maintained, supported by PaddlePaddle, including CV, NLP, Speech, Rec, TS, big models and so on.

computer-vision cv deep-learning models natural-language-processing neural-network nlp paddlepaddle recommendation speech

Last synced: 21 Nov 2026