https://github.com/shivendrra/ava

building AVA from ex-machina; a lightweight multi-modal system from scratch, just for learning & experimentation
https://github.com/shivendrra/ava

audio-classification audio-engine audio-transformers large-language-models llm machine-learning swin-transformer transformer vision vision-engine vision-models vision-transformer

Last synced: about 1 year ago
JSON representation

building AVA from ex-machina; a lightweight multi-modal system from scratch, just for learning & experimentation

• Host: GitHub
• URL: https://github.com/shivendrra/ava
• Owner: shivendrra
• License: mit
• Created: 2024-03-09T06:55:15.000Z (over 2 years ago)
• Default Branch: main
• Last Pushed: 2024-07-31T06:46:03.000Z (almost 2 years ago)
• Last Synced: 2024-12-13T20:43:03.580Z (over 1 year ago)
• Topics: audio-classification, audio-engine, audio-transformers, large-language-models, llm, machine-learning, swin-transformer, transformer, vision, vision-engine, vision-models, vision-transformer
• Language: Jupyter Notebook
• Homepage:
• Size: 11.8 MB
• Stars: 0
• Watchers: 3
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# AVA

![ava](https://liketowatchblog.files.wordpress.com/2015/10/ex-machina-ava-dress.jpg)

## Introduction

Building Ava from Ex-Machina using Language model paired audio engine to generate speech along with a vision model capabale of understanding human emotions &  . Using MoE to generated and understand speech, and then using vision models to identify and see physical things.

Trained models can be downloaded from: [huggingface/ava-v1](https://huggingface.co/shivendrra/ava-v1)

## Audio-Language Model

A transformer based language MoE model fused with an audio engine, that could directly be trained on audio lanugage rather than on written data without even need of text data in any way. Still experimenting, will see what happens.

## Vision Model

Not yet decided, but I'll soon.

## Contribution

Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.

Please make sure to update tests as appropriate.

## License

MIT