Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

Projects in Awesome Lists tagged with foundation-models

A curated list of projects in awesome lists tagged with foundation-models .

https://github.com/haotian-liu/llava

[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.

chatbot chatgpt foundation-models gpt-4 instruction-tuning llama llama-2 llama2 llava multi-modality multimodal vision-language-model visual-language-learning

Last synced: 16 Dec 2024

https://github.com/microsoft/unilm

Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

beit beit-3 bitnet deepnet document-ai foundation-models kosmos kosmos-1 layoutlm layoutxlm llm minilm mllm multimodal nlp pre-trained-model textdiffuser trocr unilm xlm-e

Last synced: 16 Dec 2024

https://github.com/haotian-liu/LLaVA

[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.

chatbot chatgpt foundation-models gpt-4 instruction-tuning llama llama-2 llama2 llava multi-modality multimodal vision-language-model visual-language-learning

Last synced: 25 Oct 2024

https://github.com/luodian/otter

🦦 Otter, a multi-modal model based on OpenFlamingo (open-sourced version of DeepMind's Flamingo), trained on MIMIC-IT and showcasing improved instruction-following and in-context learning ability.

artificial-inteligence chatgpt deep-learning embodied-ai foundation-models gpt-4 instruction-tuning large-scale-models machine-learning multi-modality visual-language-learning

Last synced: 19 Dec 2024

https://github.com/Luodian/Otter

🦦 Otter, a multi-modal model based on OpenFlamingo (open-sourced version of DeepMind's Flamingo), trained on MIMIC-IT and showcasing improved instruction-following and in-context learning ability.

artificial-inteligence chatgpt deep-learning embodied-ai foundation-models gpt-4 instruction-tuning large-scale-models machine-learning multi-modality visual-language-learning

Last synced: 24 Oct 2024

https://github.com/next-gpt/next-gpt

Code and models for NExT-GPT: Any-to-Any Multimodal Large Language Model

chatgpt foundation-models gpt-4 instruction-tuning large-language-models llm multi-modal-chatgpt multimodal visual-language-learning

Last synced: 18 Dec 2024

https://github.com/NExT-GPT/NExT-GPT

Code and models for NExT-GPT: Any-to-Any Multimodal Large Language Model

chatgpt foundation-models gpt-4 instruction-tuning large-language-models llm multi-modal-chatgpt multimodal visual-language-learning

Last synced: 24 Oct 2024

https://github.com/opengvlab/ask-anything

[CVPR2024 Highlight][VideoChatGPT] ChatGPT with video understanding! And many more supported LMs such as miniGPT4, StableLM, and MOSS.

big-model captioning-videos chat chatgpt foundation-models gradio langchain large-language-models large-model stablelm video video-question-answering video-understanding

Last synced: 18 Dec 2024

https://github.com/cluebenchmark/superclue

SuperCLUE: δΈ­ζ–‡ι€šη”¨ε€§ζ¨‘εž‹η»Όεˆζ€§εŸΊε‡† | A Benchmark for Foundation Models in Chinese

chatgpt chinese evaluation foundation-models gpt-4

Last synced: 20 Dec 2024

https://github.com/CLUEbenchmark/SuperCLUE

SuperCLUE: δΈ­ζ–‡ι€šη”¨ε€§ζ¨‘εž‹η»Όεˆζ€§εŸΊε‡† | A Benchmark for Foundation Models in Chinese

chatgpt chinese evaluation foundation-models gpt-4

Last synced: 28 Oct 2024

https://github.com/OpenGVLab/Ask-Anything

[CVPR2024 Highlight][VideoChatGPT] ChatGPT with video understanding! And many more supported LMs such as miniGPT4, StableLM, and MOSS.

big-model captioning-videos chat chatgpt foundation-models gradio langchain large-language-models large-model stablelm video video-question-answering video-understanding

Last synced: 29 Oct 2024

https://github.com/baaivision/eva

EVA Series: Visual Representation Fantasies from BAAI

foundation-models representation-learning vision-transformer

Last synced: 19 Dec 2024

https://github.com/baaivision/EVA

EVA Series: Visual Representation Fantasies from BAAI

foundation-models representation-learning vision-transformer

Last synced: 28 Oct 2024

https://github.com/deepseek-ai/deepseek-vl

DeepSeek-VL: Towards Real-World Vision-Language Understanding

foundation-models vision-language-model vision-language-pretraining

Last synced: 21 Dec 2024

https://github.com/deepseek-ai/DeepSeek-VL

DeepSeek-VL: Towards Real-World Vision-Language Understanding

foundation-models vision-language-model vision-language-pretraining

Last synced: 05 Nov 2024

https://github.com/kaiyangzhou/coop

Prompt Learning for Vision-Language Models (IJCV'22, CVPR'22)

foundation-models multimodal-learning prompt-learning

Last synced: 20 Dec 2024

https://github.com/KaiyangZhou/CoOp

Prompt Learning for Vision-Language Models (IJCV'22, CVPR'22)

foundation-models multimodal-learning prompt-learning

Last synced: 27 Oct 2024

https://github.com/tatsu-lab/alpaca_eval

An automatic evaluator for instruction-following language models. Human-validated, high-quality, cheap, and fast.

deep-learning evaluation foundation-models instruction-following large-language-models leaderboard nlp rlhf

Last synced: 17 Dec 2024

https://github.com/deepseek-ai/janus

Janus-Series: Unified Multimodal Understanding and Generation Models

any-to-any foundation-models llm multimodal unified-model vision-language-pretraining

Last synced: 20 Dec 2024

https://github.com/deepseek-ai/Janus

Janus-Series: Unified Multimodal Understanding and Generation Models

any-to-any foundation-models llm multimodal unified-model vision-language-pretraining

Last synced: 06 Dec 2024

https://tatsu-lab.github.io/alpaca_eval/

An automatic evaluator for instruction-following language models. Human-validated, high-quality, cheap, and fast.

deep-learning evaluation foundation-models instruction-following large-language-models leaderboard nlp rlhf

Last synced: 28 Oct 2024

https://github.com/OFA-Sys/ONE-PEACE

A general representation model across vision, audio, language modalities. Paper: ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalities

audio-language contrastive-loss foundation-models multimodal representation-learning vision-and-language vision-language vision-transformer

Last synced: 29 Nov 2024

https://github.com/hazyresearch/meerkat

Creative interactive views of any dataset.

data-science foundation-models machine-learning ml pandas

Last synced: 15 Dec 2024

https://github.com/HazyResearch/meerkat

Creative interactive views of any dataset.

data-science foundation-models machine-learning ml pandas

Last synced: 29 Oct 2024

https://github.com/NVlabs/FasterViT

[ICLR 2024] Official PyTorch implementation of FasterViT: Fast Vision Transformers with Hierarchical Attention

ade20k backbone coco deep-learning foundation-models image-classification image-net object-detection pre-trained-model self-attention semantic-segmentation vision-transformer visual-recognition

Last synced: 28 Oct 2024

https://github.com/mrgiovanni/modelsgenesis

[MICCAI 2019 Young Scientist Award] [MEDIA 2020 Best Paper Award] Models Genesis

3d-model fine-tuning foundation-models pre-trained-model representation-learning self-supervised-learning transfer-learning

Last synced: 16 Dec 2024

https://github.com/hazyresearch/hyena-dna

Official implementation for HyenaDNA, a long-range genomic foundation model built with Hyena

foundation-models genomics language-models

Last synced: 15 Dec 2024

https://github.com/OpenRobotLab/PointLLM

[ECCV 2024 Best Paper Candidate] PointLLM: Empowering Large Language Models to Understand Point Clouds

3d chatbot foundation-models gpt-4 large-language-models llama multimodal objaverse point-cloud pointllm representation-learning vision-and-language

Last synced: 28 Oct 2024

https://github.com/foundationvision/groma

[ECCV2024] Grounded Multimodal Large Language Model with Localized Visual Tokenization

foundation-models grounding large-language-models llama llama2 llm mllm multimodal vision-language-model

Last synced: 21 Dec 2024

https://github.com/baaivision/tokenize-anything

[ECCV 2024] Tokenize Anything via Prompting

foundation-models multimodal promptable representation-learning

Last synced: 21 Dec 2024

https://github.com/mbzuai-oryx/groundinglmm

Grounding Large Multimodal Model (GLaMM), the first-of-its-kind model capable of generating natural language responses that are seamlessly integrated with object segmentation masks [CVPR 2024].

foundation-models llm-agent lmm vision-and-language vision-language-model

Last synced: 21 Dec 2024

https://mbzuai-oryx.github.io/groundingLMM/

Grounding Large Multimodal Model (GLaMM), the first-of-its-kind model capable of generating natural language responses that are seamlessly integrated with object segmentation masks [CVPR 2024].

foundation-models llm-agent lmm vision-and-language vision-language-model

Last synced: 30 Nov 2024

https://github.com/zubair-irshad/awesome-robotics-3d

A curated list of 3D Vision papers relating to Robotics domain in the era of large models i.e. LLMs/VLMs, inspired by awesome-computer-vision, including papers, codes, and related websites

3d benchmarks computer-vision diffusion-models foundation-models gaussian-splatting grasping llm manipulation navigation nerf pointclouds policy-learning pretraining robotics scene-graph simulations vision-language-model vlm

Last synced: 16 Nov 2024

https://github.com/baaivision/uni3d

[ICLR'24 Spotlight] Uni3D: 3D Visual Representation from BAAI

3d-representation-learning foundation-models vision-transformers

Last synced: 15 Dec 2024

https://github.com/baaivision/Uni3D

[ICLR'24 Spotlight] Uni3D: 3D Visual Representation from BAAI

3d-representation-learning foundation-models vision-transformers

Last synced: 28 Oct 2024

https://github.com/wisconsinaivision/vip-llava

[CVPR2024] ViP-LLaVA: Making Large Multimodal Models Understand Arbitrary Visual Prompts

chatbot clip cvpr2024 foundation-models gpt-4 gpt-4-vision llama llama2 llava multi-modal vision-language visual-prompting

Last synced: 15 Dec 2024

https://github.com/microsoft/aurora

Implementation of the Aurora model for Earth system forecasting

atmospheric-chemistry atmospheric-dynamics aurora-model deep-learning foundation-models

Last synced: 21 Dec 2024

https://github.com/Haiyang-W/GiT

[ECCV2024 OralπŸ”₯] Official Implementation of "GiT: Towards Generalist Vision Transformer through Universal Language Interface"

foundation-models perception transformer unified vision-and-language vision-transformer

Last synced: 28 Oct 2024

https://github.com/huangwl18/language-planner

Official Code for "Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents"

artificial-intelligence codex deep-learning embodied-ai foundation-models gpt-3 in-context-learning knowledge-extraction language-model planning transformers

Last synced: 07 Nov 2024

https://github.com/aws-samples/foundation-model-benchmarking-tool

Foundation model benchmarking tool. Run any model on any AWS platform and benchmark for performance across instance type and serving stack options.

bedrock benchmark benchmarking evaluation-metrics foundation-models g5 g6 g6e generative-ai inferentia llama2 llama3 p4d p5 sagemaker trainium

Last synced: 21 Dec 2024

https://github.com/azure/intelligent-app-workshop

Immersive workshop showcasing the remarkable potential of integrating SoTA foundation models to enhance product experiences and streamline backend workflows. Leverages Microsoft's Copilot stack, Semantic Kernel and Azure primitives to offer an engaging and comprehensive introduction to AI-infused app development and deployment

ai foundation-models gpt-35-turbo intelligent-agents intelligent-app llm ml mlops prompt-engineering semantic-kernel

Last synced: 21 Dec 2024

https://github.com/Azure/intelligent-app-workshop

Immersive workshop showcasing the remarkable potential of integrating SoTA foundation models to enhance product experiences and streamline backend workflows. Leverages Microsoft's Copilot stack, Semantic Kernel and Azure primitives to offer an engaging and comprehensive introduction to AI-infused app development and deployment

ai foundation-models gpt-35-turbo intelligent-agents intelligent-app llm ml mlops prompt-engineering semantic-kernel

Last synced: 04 Nov 2024

https://github.com/zjysteven/lmms-finetune

A minimal codebase for finetuning large multimodal models, supporting llava-1.5/1.6, llava-interleave, llava-next-video, llava-onevision, qwen-vl, qwen2-vl, phi3-v etc.

finetuning foundation-models instruction-tuning large-language-model large-multimodal-models llava llava-next multimodal multimodal-large-language-models qwen-vl vision-language visual-instruction-tuning

Last synced: 21 Dec 2024

https://github.com/om-ai-lab/rs5m

RS5M: a large-scale vision language dataset for remote sensing

foundation-models remote-sensing vision-and-language

Last synced: 06 Nov 2024

https://github.com/om-ai-lab/RS5M

RS5M: a large-scale vision language dataset for remote sensing

foundation-models remote-sensing vision-and-language

Last synced: 05 Nov 2024

https://github.com/vitae-transformer/mtp

The official repo for "MTP: Advancing Remote Sensing Foundation Model via Multi-Task Pretraining"

change-detection classification deep-learning foundation-models object-detection pre-training remote-sensing semantic-segmentation transfer-learning

Last synced: 24 Nov 2024

https://github.com/ViTAE-Transformer/MTP

The official repo for "MTP: Advancing Remote Sensing Foundation Model via Multi-Task Pretraining"

change-detection classification deep-learning foundation-models object-detection pre-training remote-sensing semantic-segmentation transfer-learning

Last synced: 05 Nov 2024

https://github.com/OxWearables/ssl-wearables

Self-supervised learning for wearables using the UK-Biobank (>700,000 person-days)

accelerometer deep-learning foundation-models human-activity-recognition pytorch self-supervised-learning wearable

Last synced: 06 Nov 2024

https://github.com/mazurowski-lab/finetune-SAM

This is an official repo for fine-tuning SAM to customized medical images.

finetune foundation-models medical-imaging sam

Last synced: 30 Nov 2024

https://github.com/aim-harvard/foundation-cancer-image-biomarker

[Nature Machine Intelligence 2024] Code and evaluation repository for the paper

cancer-imaging-research foundation-models medical-imaging representation-learning simclr

Last synced: 21 Dec 2024

https://github.com/microsoft/dpsda

Private Evolution: Generating DP Synthetic Data without Training [ICLR 2024, ICML 2024]

differential-privacy foundation-models private-evolution synthetic-data training-free

Last synced: 17 Dec 2024

https://github.com/microsoft/DPSDA

[ICLR 2024] Generating DP Synthetic Data without Training

differential-privacy foundation-models synthetic-data training-free

Last synced: 05 Nov 2024

https://github.com/sayakpaul/robustness-foundation-models

This repository holds code and other relevant files for the NeurIPS 2022 tutorial: Foundational Robustness of Foundation Models.

foundation-models representation-learning robustness

Last synced: 09 Nov 2024

https://github.com/jieyuz2/taskmeanything

[NeurIPS 2024] A task generation and model evaluation system for multimodal language models.

benchmark evaluation foundation-models

Last synced: 18 Dec 2024

https://github.com/yasserben/CLOUDS

[CVPR 2024] Official Implementation of Collaborating Foundation models for Domain Generalized Semantic Segmentation

deep-learning detectron2 domain-adaptation domain-generalization foundation-models mask2former semantic-segmentation transformer

Last synced: 30 Nov 2024

https://github.com/SaberaTalukder/TOTEM

The official code πŸ‘©β€πŸ’» for - TOTEM: TOkenized Time Series EMbeddings for General Time Series Analysis

foundation-models representation-learning time-series time-series-analysis time-series-anomaly-detection time-series-forecasting time-series-foundation-model time-series-imputation tokenization

Last synced: 30 Aug 2024

https://github.com/kaiko-ai/eva

Evaluation framework for oncology foundation models (FMs)

evaluation-framework foundation-models machine-learning oncology

Last synced: 13 Nov 2024

https://github.com/rhysdg/vision-at-a-clip

Low-latency ONNX and TensorRT based zero-shot classification and detection with contrastive language-image pre-training based prompts

clip foundation-models grounding-dino machine-learning onnx siglip tensorrt zero-shot-classification zero-shot-object-detection

Last synced: 27 Oct 2024

https://github.com/neuraloperator/coda-no

Codomain attention neural operator for single to multi-physics PDE adaptation.

foundation-models neural-operator pde-solver

Last synced: 09 Nov 2024

https://github.com/pnnl/cactus

LLM Agent that leverages cheminformatics tools to provide informed responses.

cheminformatics chemistry foundation-models llm llm-agent nlp science

Last synced: 25 Nov 2024

https://github.com/noodlefrenzy/promptgen

CLI for managing and generating Foundation Model prompts

chatgpt foundation-models gpt llms midjourney stable-diffusion

Last synced: 07 Nov 2024

https://github.com/wkentaro/yolo-world-onnx

ONNX models of YOLO-World (an open-vocabulary object detection).

computer-vision deep-learning foundation-models object-detection

Last synced: 08 Nov 2024

https://github.com/build-on-aws/amazon-bedrock-with-builder-and-command-patterns

A simple, yet powerful implementation in Java that allows developers to write a rather straightforward code to create the API requests for the different foundation models supported by Amazon Bedrock.

amazon bedrock builder-pattern command-pattern foundation-models generative-ai java llm

Last synced: 07 Nov 2024

https://github.com/fedebotu/green-planet-transformers-3

MelXior: a Neural Weather Forecasting app distilling knowledge from models including GPT-3, DALL-E and FourCastNet

climate dalle2 foundation-models fourcastnet gpt3 hackaton openai transformers weather

Last synced: 06 Nov 2024

https://github.com/techthoughts2/pwshbedrock

pwshBedrock is a PowerShell module designed to simplify interaction with Amazon Bedrock foundation models. It enables users to send messages, retrieve responses, manage conversation contexts, generate images, and estimate costs. Supporting both InvokeModel and Converse API, it streamlines AI integration in PowerShell workflows.

ai ai21labs amazon-bedrock amazon-titan amazon-web-services anthropic-claude aws claude-3 cohere command-r command-r-plus foundation-models generative-ai jamba large-language-models meta-llama3 mistral-ai powershell powershell-module stability-ai

Last synced: 03 Dec 2024

https://github.com/jiayuww/spatialeval

[NeurIPS'24] SpatialEval: a benchmark to evaluate spatial reasoning abilities of MLLMs and LLMs

claude foundation-models gemini gpt-4o gpt-4v large-language-models llama3 machine-learning multimodal-deep-learning reasoning spatial-reasoning vision-language-models

Last synced: 03 Dec 2024

https://github.com/ycheng517/awesome-foundation-model-ros

A collection ROS projects utilizing foundation models.

awesome awesome-list foundation-models robotics ros ros2

Last synced: 29 Oct 2024

https://github.com/mbari-org/aipipeline

Library for running detection, clustering or classification ai pipelines plus performance monitoring

foundation-models image-classification object-detection

Last synced: 10 Dec 2024

https://github.com/mbari-org/fastapi-vss

RESTful API for vector similarity search. It uses the Python web framework FastAPI. This accelerates machine learning workflows that require vector similarity search using foundational models.

fastapi foundation-models image-classification vision-transformer

Last synced: 10 Dec 2024

https://github.com/microsoft/mattersim

MatterSim: A deep learning atomistic model across elements, temperatures and pressures.

ai4materials ai4science computational-materials-science foundation-models machine-learning machine-learning-force-field materials-science mlff

Last synced: 03 Dec 2024

https://github.com/agora-lab-ai/forestnet

A Deep Learning Framework for Quantifying Collective Forest Intelligence Through Multi-Variable Temporal-Spatial Analysis

ai amazon amazonforests bioinformatics biology biologyai bioml collective-behavior collective-intelligence forest-ai forests foundation-models greenery greenery-ai ml swarms

Last synced: 10 Nov 2024

https://github.com/himanshuvnm/foundation-model-large-language-model-fm-llm

This repository was commited under the action of executing important tasks on which modern Generative AI concepts are laid on. In particular, we focussed on three coding actions of Large Language Models. Extra and necessary details are given in the README.md file.

attention-is-all-you-need aws fine-tuning flan-t5 foundation-models generative-ai hate-speech-detection huggingface huggingface-transformers large-language-models lora low-rank-ada ml-m5-2xlarge peft-fine-tuning-llm python3 pytorch rlhf rnn-pytorch

Last synced: 06 Nov 2024