https://github.com/kyegomez/celestial-1
Omni-Modality Processing, Understanding, and Generation
https://github.com/kyegomez/celestial-1
attention attention-is-all-you-need attention-mechanisms gpt-4 gpt4 multi-modal multimodal multimodal-deep-learning multimodality omnimodal openai
Last synced: 4 months ago
JSON representation
Omni-Modality Processing, Understanding, and Generation
- Host: GitHub
- URL: https://github.com/kyegomez/celestial-1
- Owner: kyegomez
- License: apache-2.0
- Created: 2023-05-20T02:47:33.000Z (about 3 years ago)
- Default Branch: master
- Last Pushed: 2024-05-03T21:27:46.000Z (about 2 years ago)
- Last Synced: 2025-05-07T07:47:05.202Z (about 1 year ago)
- Topics: attention, attention-is-all-you-need, attention-mechanisms, gpt-4, gpt4, multi-modal, multimodal, multimodal-deep-learning, multimodality, omnimodal, openai
- Language: Python
- Homepage: https://discord.gg/qUtxnK2NMf
- Size: 2.49 MB
- Stars: 7
- Watchers: 3
- Forks: 1
- Open Issues: 4
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
[](https://discord.gg/qUtxnK2NMf)
# CELESTIAL-1: A Model for Processing, Understanding, and Reasoning in Unlimited Modalities
[](https://github.com/kyegomez/CELESTIAL-1/issues)
[](https://github.com/kyegomez/CELESTIAL-1/network)
[](https://github.com/kyegomez/CELESTIAL-1/stargazers) [](https://github.com/kyegomez/CELESTIAL-1/blob/master/LICENSE)
[](https://twitter.com/intent/tweet?text=Excited%20to%20introduce%20CELESTIAL-1,%20the%20all-new%20Multi-Modal%20model%20with%20the%20potential%20to%20revolutionize%20automation.%20Join%20us%20on%20this%20journey%20towards%20a%20smarter%20future.%20%23CELESTIAL-1%20%23Multi-Modal&url=https%3A%2F%2Fgithub.com%2Fkyegomez%2FCELESTIAL-1)
[](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Fgithub.com%2Fkyegomez%2FCELESTIAL-1)
[](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Fgithub.com%2Fkyegomez%2FCELESTIAL-1&title=Introducing%20CELESTIAL-1%2C%20the%20All-New%20Multi-Modal%20Model&summary=CELESTIAL-1%20is%20the%20next-generation%20Multi-Modal%20model%20that%20promises%20to%20transform%20industries%20with%20its%20intelligence%20and%20efficiency.%20Join%20us%20to%20be%20a%20part%20of%20this%20revolutionary%20journey%20%23CELESTIAL-1%20%23Multi-Modal&source=)
[](https://www.reddit.com/submit?url=https%3A%2F%2Fgithub.com%2Fkyegomez%2FCELESTIAL-1&title=Exciting%20Times%20Ahead%20with%20CELESTIAL-1%2C%20the%20All-New%20Multi-Modal%20Model%20%23CELESTIAL-1%20%23Multi-Modal) [](https://news.ycombinator.com/submitlink?u=https%3A%2F%2Fgithub.com%2Fkyegomez%2FCELESTIAL-1&t=Exciting%20Times%20Ahead%20with%20CELESTIAL-1%2C%20the%20All-New%20Multi-Modal%20Model%20%23CELESTIAL-1%20%23Multi-Modal)
[](https://pinterest.com/pin/create/button/?url=https%3A%2F%2Fgithub.com%2Fkyegomez%2FCELESTIAL-1&media=https%3A%2F%2Fexample.com%2Fimage.jpg&description=CELESTIAL-1%2C%20the%20Revolutionary%20Multi-Modal%20Model%20that%20will%20Change%20the%20Way%20We%20Work%20%23CELESTIAL-1%20%23Multi-Modal)
[](https://api.whatsapp.com/send?text=I%20just%20discovered%20CELESTIAL-1,%20the%20all-new%20Multi-Modal%20model%20that%20promises%20to%20revolutionize%20automation.%20Join%20me%20on%20this%20exciting%20journey%20towards%20a%20smarter%20future.%20%23CELESTIAL-1%20%23Multi-Modal%0A%0Ahttps%3A%2F%2Fgithub.com%2Fkyegomez%2FCELESTIAL-1)
CELESTIAL-1 is an open source model designed to provide a general representation for processing, understanding, and reasoning across an unlimited range of modalities.
CELESTIAL-1 can operate in vision, audio, audio-language, and vision-language tasks without any prior initialization.
This model will change everything for our species, we'll able to automate any digital task. And, we need your help. [Contribute to the project board here:](https://github.com/users/kyegomez/projects/6/views/2)
# Install
`pip install cstx`
# Usage
## Features
CELESTIAL-1 boasts a wide range of features designed to enhance its versatility and usefulness. Some of these features include:
* **Unlimited Modalities**: The model can process and understand an unlimited number of modalities, increasing its applicability in a wide range of tasks.
* **Reasoning in Context**: CELESTIAL-1 is capable of reasoning in context across multiple modalities, thus improving its understanding and representation of complex scenarios.
* **Generates Any Modality**: With the ability to generate any modality, CELESTIAL-1 can cater to a wider variety of applications.
* **Scaling-Friendly Architecture**: The architecture of CELESTIAL-1 is designed to be friendly to scaling, which allows it to efficiently handle larger and more complex tasks.
* **Modality-Agnostic Tasks**: The model can perform tasks that are agnostic to the modality, further widening its range of uses.
* **Strong Emergent Zero-Shot Retrieval Capability**: CELESTIAL-1 has an impressive ability to retrieve information in a zero-shot manner, an essential feature for any advanced AI model.
---
## Contribution
Contributing to CELESTIAL-1 is a rewarding experience, whether you're a novice developer or an AI expert. Here's a simple guide to get you started:
1. **Understand the Project**:
Familiarize yourself with CELESTIAL-1's mission and features. Check out the project board: [Contribute to the project board](https://github.com/users/kyegomez/projects/6/views/2).
2. **Choose a Task**:
Select a task that resonates with your skill set and passion. It could range from documentation, coding, testing, or even discussions.
3. **Fork the Repository**:
Create a personal fork of the CELESTIAL-1 repository on GitHub.
4. **Clone Your Fork**:
```bash
git clone https://github.com//Celestial-1.git
```
5. **Branching**:
Always create a new branch for each task. This keeps things organized.
```bash
git checkout -b feature/your-branch-name
```
6. **Code, Document, Discuss**:
Dive in! Make your changes, write/document, or participate in discussions.
7. **Commit and Push**:
After making your changes, commit them and push them to your fork.
```bash
git commit -m "Your concise commit message"
git push origin feature/your-branch-name
```
8. **Create a Pull Request**:
Go to the original CELESTIAL-1 repository on GitHub and create a pull request. Fill out the provided template with as much detail as possible.
9. **Wait for Review**:
Maintainers or other contributors will review your changes. They might ask for edits or further information. Collaboration is key!
10. **Celebrate**:
Once your contribution is merged, celebrate! You've just made an impactful contribution to an incredible project.
## Tips:
- Always sync your fork with the main repository before starting on a new feature.
- Clearly document any changes or additions you make.
- Participate actively in discussions and respect all contributors.
Remember, every contribution, no matter how small, is valuable. Your insights, skills, and time are greatly appreciated. Let's make CELESTIAL-1 remarkable together!
---
## References
For more information about the ONE-PEACE architecture that CELESTIAL-1 is based on, you can refer to the following resources:
* ONE-PEACE Paper: [Link](#)
* ONE-PEACE Demo: [Link](#)
* ONE-PEACE Checkpoints: [Link](#)
[Vicuna](https://github.com/lm-sys/FastChat),
[ImageBind](https://github.com/facebookresearch/ImageBind),
[Stable Diffusion](https://huggingface.co/docs/diffusers/api/pipelines/stable_diffusion/text2img),
[AudioLDM](https://github.com/haoheliu/AudioLDM)
[Zeroscope](https://huggingface.co/cerspense/zeroscope_v2_576w).
[PandaGPT](https://github.com/yxuansu/PandaGPT),
[VPGTrans](https://vpgtrans.github.io/),
[GILL](https://github.com/kohjingyu/gill/),
[CoDi](https://codi-gen.github.io/),
[Video-LLaMA](https://github.com/DAMO-NLP-SG/Video-LLaMA),
and [MiniGPT-4](https://github.com/Vision-CAIR/MiniGPT-4)
-------
## License
CELESTIAL-1 is released under the Apache 2.0 license. For more information about what this license entails, please refer to the LICENSE file.