Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
Projects in Awesome Lists tagged with pre-training
A curated list of projects in awesome lists tagged with pre-training .
https://github.com/rucaibox/llmsurvey
The official GitHub page for the survey paper "A Survey of Large Language Models".
chain-of-thought chatgpt in-context-learning instruction-tuning large-language-models llm llms natural-language-processing pre-trained-language-models pre-training rlhf
Last synced: 29 Sep 2024
https://github.com/RUCAIBox/LLMSurvey
The official GitHub page for the survey paper "A Survey of Large Language Models".
chain-of-thought chatgpt in-context-learning instruction-tuning large-language-models llm llms natural-language-processing pre-trained-language-models pre-training rlhf
Last synced: 30 Jul 2024
https://github.com/dbiir/uer-py
Open Source Pre-training Model Framework in PyTorch & Pre-trained Model Zoo
albert bart bert chinese classification clue elmo fine-tuning gpt gpt-2 model-zoo natural-language-processing ner pegasus pre-training pytorch roberta t5 unilm xlm-roberta
Last synced: 30 Sep 2024
https://github.com/dbiir/UER-py
Open Source Pre-training Model Framework in PyTorch & Pre-trained Model Zoo
albert bart bert chinese classification clue elmo fine-tuning gpt gpt-2 model-zoo natural-language-processing ner pegasus pre-training pytorch roberta t5 unilm xlm-roberta
Last synced: 01 Aug 2024
https://github.com/modelscope/ms-swift
Use PEFT or Full-parameter to finetune 300+ LLMs or 50+ MLLMs. (Qwen2, GLM4v, Internlm2.5, Yi, Llama3.1, Llava-Video, Internvl2, MiniCPM-V-2.6, Deepseek, Baichuan2, Gemma2, Phi3-Vision, ...)
agent deploy dpo glm4 internvl llama llama3 llava llm lora megatron minicpm-v modelscope multimodal ollama peft pre-training qwen2 sft unsloth
Last synced: 27 Sep 2024
https://github.com/modelscope/swift
ms-swift: Use PEFT or Full-parameter to finetune 300+ LLMs or 50+ MLLMs. (Qwen2, GLM4v, Internlm2.5, Yi, Llama3.1, Llava-Video, Internvl2, MiniCPM-V, Deepseek, Baichuan2, Gemma2, Phi3-Vision, ...)
agent deploy dpo glm4 internvl llama llama3 llava llm lora megatron mistral-nemo modelscope multimodal ollama peft pre-training qwen2 sft unsloth
Last synced: 02 Aug 2024
https://github.com/modelscope/data-juicer
A one-stop data processing system to make data higher-quality, juicier, and more digestible for (multimodal) LLMs! ๐ ๐ ๐ฝ โก๏ธ โก๏ธ๐ธ ๐น ๐ทไธบๅคงๆจกๅๆไพๆด้ซ่ดจ้ใๆดไธฐๅฏใๆดๆโๆถๅโ็ๆฐๆฎ๏ผ
chinese data-analysis data-science data-visualization dataset gpt gpt-4 instruction-tuning large-language-models llama llava llm llms multi-modal nlp opendata pre-training pytorch sora streamlit
Last synced: 29 Sep 2024
https://github.com/EgoAlpha/prompt-in-context-learning
Awesome resources for in-context learning and prompt engineering: Mastery of the LLMs such as ChatGPT, GPT-3, and FlanT5, with up-to-date and cutting-edge updates.
chain-of-thought chatbot chatgpt chatgpt-api cot in-context-learning language-modeling language-understanding large-language-model llm pre-training prompt prompt-based-learning prompt-design prompt-engineering prompt-learning prompt-toolkit prompt-tuning
Last synced: 29 Jul 2024
https://github.com/egoalpha/prompt-in-context-learning
Awesome resources for in-context learning and prompt engineering: Mastery of the LLMs such as ChatGPT, GPT-3, and FlanT5, with up-to-date and cutting-edge updates.
chain-of-thought chatbot chatgpt chatgpt-api cot in-context-learning language-modeling language-understanding large-language-model llm pre-training prompt prompt-based-learning prompt-design prompt-engineering prompt-learning prompt-toolkit prompt-tuning
Last synced: 30 Sep 2024
https://github.com/microsoft/oscar
Oscar and VinVL
image-captioning image-text-search oscar pre-training vinvl vision-and-language vqa
Last synced: 26 Sep 2024
https://github.com/microsoft/Oscar
Oscar and VinVL
image-captioning image-text-search oscar pre-training vinvl vision-and-language vqa
Last synced: 08 Aug 2024
https://github.com/zjunlp/knowlm
An Open-sourced Knowledgable Large Language Model Framework.
bilingual chinese deep-learning deepspeed english gpt-3 instructie instruction-following instruction-tuning instructions knowlm language-model large-language-models llama lora models pre-trained-language-models pre-trained-model pre-training reasoning
Last synced: 27 Sep 2024
https://github.com/zjunlp/KnowLM
An Open-sourced Knowledgable Large Language Model Framework.
bilingual chinese deep-learning deepspeed english gpt-3 instructie instruction-following instruction-tuning instructions knowlm language-model large-language-models llama lora models pre-trained-language-models pre-trained-model pre-training reasoning
Last synced: 01 Aug 2024
https://github.com/brightmart/bert_language_understanding
Pre-training of Deep Bidirectional Transformers for Language Understanding: pre-train TextCNN
attention-is-all-you-need bert-model document-classification fasttext language-model language-understanding nlp pre-training question-answering self-attention text-classification textcnn transfer-learning transformer-encoder
Last synced: 01 Aug 2024
https://github.com/jackroos/VL-BERT
Code for ICLR 2020 paper "VL-BERT: Pre-training of Generic Visual-Linguistic Representations".
bert iclr2020 pre-training pytorch representation-learning self-supervised-learning vision-and-language vl-bert
Last synced: 01 Aug 2024
https://github.com/princeton-nlp/llm-shearing
[ICLR 2024] Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning
efficiency llama llama2 llm nlp pre-training pruning
Last synced: 27 Sep 2024
https://github.com/Shen-Lab/GraphCL
[NeurIPS 2020] "Graph Contrastive Learning with Augmentations" by Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, Yang Shen
contrastive-learning graph-neural-network pre-training self-supervised-learning
Last synced: 08 Aug 2024
https://github.com/princeton-nlp/LLM-Shearing
[ICLR 2024] Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning
efficiency llama llama2 llm nlp pre-training pruning
Last synced: 01 Aug 2024
https://github.com/acbull/GPT-GNN
Code for KDD'20 "Generative Pre-Training of Graph Neural Networks"
graph-neural-networks graph-representation-learning pre-training self-supervised-learning
Last synced: 08 Aug 2024
https://github.com/microsoft/XPretrain
Multi-modality pre-training
computer-vision multimedia multimodal-learning nlp pre-training
Last synced: 01 Aug 2024
https://github.com/GAIR-NLP/MathPile
Generative AI for Math: MathPile
corpus language-model large-language-models math pre-training
Last synced: 09 Aug 2024
https://github.com/THUDM/GCC
GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training @ KDD 2020
contrastive-learning graph-neural-networks pre-training
Last synced: 08 Aug 2024
https://github.com/sayakpaul/probing-vits
Probing the representations of Vision Transformers.
attention explaining-vits image-recognition keras pre-training self-supervision tensorflow transformers vits
Last synced: 03 Oct 2024
https://github.com/westlake-repl/recommendation-systems-without-explicit-id-features-a-literature-review
Paper List of Pre-trained Foundation Recommender Models
chatgpt chatgpt3 chatgpt4rec cross-domain-recommendation cross-domainrecommendation foundation-model gpt4rec language-model large-language-model llm llm-recommendation llm4rec multimodal multimodal-deep-learning multimodalrecommendation pre-training recommendation-system recommender-system transfer-learning transferable
Last synced: 02 Aug 2024
https://github.com/wangxiao5791509/MultiModal_BigModels_Survey
[MIR-2023-Survey] A continuously updated paper list for multi-modal pre-trained big models
anhui-university audio big-models depth event-camera multi-modal natural-language pengchenglab point-cloud pre-training radar review rgb-text-audio self-attention survey thermal-infrared transformers
Last synced: 01 Aug 2024
https://github.com/ViTAE-Transformer/SAMRS
The official repo for [NeurIPS'23] "SAMRS: Scaling-up Remote Sensing Segmentation Dataset with Segment Anything Model"
dataset deep-learning pre-training remote-sensing sam segment-anything-model semantic-segmentation transfer-learning
Last synced: 31 Jul 2024
https://github.com/akanyaani/gpt-2-tensorflow2.0
OpenAI GPT2 pre-training and sequence prediction implementation in Tensorflow 2.0
gpt gpt-2 gpt2 implementation nlp openai pre-training pretraining tensorflow tensorflow2 text-generation transformer
Last synced: 02 Aug 2024
https://github.com/DeepGraphLearning/GearNet
GearNet and Geometric Pretraining Methods for Protein Structure Representation Learning, ICLR'2023 (https://arxiv.org/abs/2203.06125)
graph-neural-networks pre-training protein-representation-learning
Last synced: 03 Aug 2024
https://github.com/OpenDriveLab/ViDAR
[CVPR 2024 Highlight] Visual Point Cloud Forecasting
autonomous-driving point-cloud-forecasting pre-training world-model
Last synced: 31 Jul 2024
https://github.com/lucidrains/electra-pytorch
A simple and working implementation of Electra, the fastest way to pretrain language models from scratch, in Pytorch
artificial-intelligence deep-learning pre-training transformer
Last synced: 03 Oct 2024
https://github.com/michiyasunaga/DrRepair
[ICML 2020] DrRepair: Learning to Repair Programs from Error Messages
code-generation deep-learning graph-neural-networks pre-training program-repair
Last synced: 09 Aug 2024
https://github.com/iamyuanchung/Autoregressive-Predictive-Coding
Autoregressive Predictive Coding: An unsupervised autoregressive model for speech representation learning
pre-training pytorch representation-learning self-supervised-learning unsupervised-learning
Last synced: 07 Aug 2024
https://github.com/balavenkatesh3322/audio-pretrained-model
A collection of Audio and Speech pre-trained models.
audio audio-processing caffe keras keras-models keras-tensorflow machine-learning mxnet neural-network pre-trained pre-trained-model pre-training python3 pytorch pytorch-models speech-recognition speech-to-text tensorflow tensorflow-models
Last synced: 29 Sep 2024
https://github.com/lucidrains/mlm-pytorch
An implementation of masked language modeling for Pytorch, made as concise and simple as possible
artificial-intelligence deep-learning pre-training transformers unsupervised-learning
Last synced: 03 Oct 2024
https://github.com/ZhangYuanhan-AI/Bamboo
Bamboo: 4 times larger than ImageNet; 2 time larger than Object365; Built by active learning.
active-learning dataset-generation pre-training
Last synced: 03 Aug 2024
https://github.com/zjunlp/molgen
[ICLR 2024] Domain-Agnostic Molecular Generation with Chemical Feedback
generation huggingface iclr2024 language-model molecular-generation molecular-optimization molecule molgen multitask pre-trained-language-models pre-trained-model pre-training pytorch selfies targeted-molecular-generation
Last synced: 26 Sep 2024
https://github.com/ViTAE-Transformer/MTP
The official repo for "MTP: Advancing Remote Sensing Foundation Model via Multi-Task Pretraining"
change-detection classification deep-learning foundation-models object-detection pre-training remote-sensing semantic-segmentation transfer-learning
Last synced: 01 Aug 2024
https://github.com/lucidrains/marge-pytorch
Implementation of Marge, Pre-training via Paraphrasing, in Pytorch
artificial-intelligence deep-learning pre-training retrieval transformers
Last synced: 03 Oct 2024
https://github.com/VITA-Group/CV_LTH_Pre-training
[CVPR 2021] "The Lottery Tickets Hypothesis for Supervised and Self-supervised Pre-training in Computer Vision Models" Tianlong Chen, Jonathan Frankle, Shiyu Chang, Sijia Liu, Yang Zhang, Michael Carbin, Zhangyang Wang
imagenet-pr lottery-ticket-hypothesis moco pre-training simclr simclrv2 transfer transfer-learning
Last synced: 03 Aug 2024
https://github.com/lucidrains/coco-lm-pytorch
Implementation of COCO-LM, Correcting and Contrasting Text Sequences for Language Model Pretraining, in Pytorch
artificial-intelligence deep-learning pre-training transformers
Last synced: 03 Oct 2024
https://github.com/fajieyuan/SIGIR2021_Conure
Pre-training and Lifelong learning for User Embedding and Recommender System
catastrophic-forgetting continual-learning foundation-model general-purpose lifelong-learning lifelong-machine-learning one-for-all pre-training self-supervised-learning transfer-learning universal-recommender universal-representation user-profile user-representation
Last synced: 02 Aug 2024
https://github.com/FudanDISC/ReForm-Eval
An benchmark for evaluating the capabilities of large vision-language models (LVLMs)
benchmark embodied-ai gpt4 in-context-learning instruction-following instruction-tuning large-language-models large-vision-language-models llm multimodal multimodal-chain-of-thought pre-training reformulation visual-chain-of-thought
Last synced: 02 Aug 2024
https://github.com/fajieyuan/universal_user_representation
papers of universal user representation learning for recommendation
bert catastrophic-forgetting cold-start continual-learning cross-domain-recommendation general-purpose item-recommendation lifelong-learning multi-domain-recommendation pre-training pruning recommender-system representation-learning transfer transfer-learning transformer trimming user-modeling user-profiling user-representation
Last synced: 28 Sep 2024
https://github.com/SLAMPAI/large-scale-pretraining-transfer
Code for reproducing the experiments on large-scale pre-training and transfer learning for the paper "Effect of large-scale pre-training on full and few-shot transfer learning for natural and medical images" (https://arxiv.org/abs/2106.00116)
big-transfer chest-x-ray14 chest-xray-images chexpert-dataset covidx-dataset deep-learning distributed-training few-shot-learning fine-tuning imagenet large-scale-learning medical-imaging mimic-cxr padchest-dataset pre-trained-model pre-training pytorch scaling-laws supercomputing transfer-learning
Last synced: 03 Aug 2024
https://github.com/sanketvmehta/lifelong-learning-pretraining-and-sam
Code for the paper "Mehta, S. V., Patil, D., Chandar, S., & Strubell, E. (2023). An Empirical Investigation of the Role of Pre-training in Lifelong Learning. The Journal of Machine Learning Research 24 (2023)"
continual-learning flat-minima lifelong-learning pre-training sharpness-aware-minimization
Last synced: 03 Aug 2024
