Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/LAION-AI/Open-Assistant

OpenAssistant is a chat-based assistant that understands tasks, can interact with third-party systems, and retrieve information dynamically to do so.
https://github.com/LAION-AI/Open-Assistant

ai assistant chatgpt discord-bot language-model machine-learning nextjs python rlhf

Last synced: about 2 months ago
JSON representation

OpenAssistant is a chat-based assistant that understands tasks, can interact with third-party systems, and retrieve information dynamically to do so.

Awesome Lists containing this project

README 

        


    Open-Assistant

  






:memo: NOTE: OpenAssistant is completed, and the project is now finished. Thank you to everyone who contributed! Check out our blog post for more information. The final published oasst2 dataset can be found on HuggingFace at OpenAssistant/oasst2








![GitHub Repo stars](https://img.shields.io/github/stars/LAION-AI/Open-Assistant?style=social)

![Docs](https://img.shields.io/badge/docs-laion--ai.github.io%2FOpen--Assistant%2F-green)

![GitHub Workflow Status](https://img.shields.io/github/actions/workflow/status/LAION-AI/Open-Assistant/build-frontend.yaml?label=build-frontend)

![GitHub Workflow Status](https://img.shields.io/github/actions/workflow/status/LAION-AI/Open-Assistant/build-postgres.yaml?label=build-postgres)

![GitHub Workflow Status](https://img.shields.io/github/actions/workflow/status/LAION-AI/Open-Assistant/pre-commit.yaml?label=pre-commit)

![GitHub Workflow Status](https://img.shields.io/github/actions/workflow/status/LAION-AI/Open-Assistant/test-api-contract.yaml?label=tests-api)

![GitHub Workflow Status](https://img.shields.io/github/actions/workflow/status/LAION-AI/Open-Assistant/test-e2e.yaml?label=tests-web)

![GitHub Workflow Status](https://img.shields.io/github/actions/workflow/status/LAION-AI/Open-Assistant/deploy-docs-site.yaml?label=deploy-docs)

![GitHub Workflow Status](https://img.shields.io/github/actions/workflow/status/LAION-AI/Open-Assistant/production-deploy.yaml?label=deploy-production)

![GitHub Workflow Status](https://img.shields.io/github/actions/workflow/status/LAION-AI/Open-Assistant/release.yaml?label=deploy-release)

![GitHub release (latest by date)](https://img.shields.io/github/v/release/LAION-AI/Open-Assistant)

![Translate](https://img.shields.io/badge/Translate-blue)






# Table of Contents



- [What is Open Assistant?](#what-is-open-assistant)

- [Useful Links](#useful-links)

- [How To Try It Out](#how-to-try-it-out)

- [The Vision](#the-vision)

- [The Plan](#the-plan)

- [How You Can Help](#how-you-can-help)



---



## What is Open Assistant?





Open Assistant is a project meant to give everyone access to a great chat based

large language model.




We believe that by doing this we will create a revolution in innovation in

language. In the same way that stable-diffusion helped the world make art and

images in new ways we hope Open Assistant can help improve the world by

improving language itself.



# Useful Links



- [Data Collection](https://open-assistant.io)



- [Chat](https://open-assistant.io/chat)



- [Project Documentation](https://projects.laion.ai/Open-Assistant/)



## How To Try It Out



### Chatting with the AI



The chat frontend is now live [here](https://open-assistant.io/chat). Log in and

start chatting! Please try to react with a thumbs up or down for the assistant's

responses when chatting.



### Contributing to Data Collection



The data collection frontend is now live [here](https://open-assistant.io/). Log

in and start taking on tasks! We want to collect a high volume of quality data.

By submitting, ranking, and labelling model prompts and responses you will be

directly helping to improve the capabilities of Open Assistant.



### Running the Development Setup Locally (without chat)



**You do not need to run the project locally unless you are contributing to the

development process. The website link above will take you to the public website

where you can use the data collection app and the chat.**



If you would like to run the data collection app locally for development, you

can set up an entire stack needed to run **Open-Assistant**, including the

website, backend, and associated dependent services, with Docker.



To start the demo, run this in the root directory of the repository (check

[this FAQ](https://projects.laion.ai/Open-Assistant/docs/faq#docker-compose-instead-of-docker-compose)

if you have problems):



```sh

docker compose --profile ci up --build --attach-dependencies

```



> **Note:** when running on MacOS with an M1 chip you have to use:

> `DB_PLATFORM=linux/x86_64 docker compose ...`



Then, navigate to `http://localhost:3000` (It may take some time to boot up) and

interact with the website.



> **Note:** If an issue occurs with the build, please head to the

> [FAQ](https://projects.laion.ai/Open-Assistant/docs/faq) and check out the

> entries about Docker.



> **Note:** When logging in via email, navigate to `http://localhost:1080` to

> get the magic email login link.



> **Note:** If you would like to run this in a standardized development

> environment (a

> ["devcontainer"](https://code.visualstudio.com/docs/devcontainers/containers))

> using

> [vscode locally](https://code.visualstudio.com/docs/devcontainers/create-dev-container#_create-a-devcontainerjson-file)

> or in a web browser using

> [GitHub Codespaces](https://github.com/features/codespaces), you can use the

> provided [`.devcontainer`](.devcontainer/) folder.



### Running the Development Setup Locally for Chat



**You do not need to run the project locally unless you are contributing to the

development process. The website link above will take you to the public website

where you can use the data collection app and the chat.**



**Also note that the local setup is only for development and is not meant to be

used as a local chatbot, unless you know what you are doing.**



If you _do_ know what you are doing, then see the `inference` folder for getting

the inference system up and running, or have a look at `--profile inference` in

addition to `--profile ci` in the above command.



## The Vision



We are not going to stop at replicating ChatGPT. We want to build the assistant

of the future, able to not only write email and cover letters, but do meaningful

work, use APIs, dynamically research information, and much more, with the

ability to be personalized and extended by anyone. And we want to do this in a

way that is open and accessible, which means we must not only build a great

assistant, but also make it small and efficient enough to run on consumer

hardware.



## The Plan



##### We want to get to an initial MVP as fast as possible, by following the 3-steps outlined in the [InstructGPT paper](https://arxiv.org/abs/2203.02155)



1. Collect high-quality human generated Instruction-Fulfillment samples

   (prompt + response), goal >50k. We design a crowdsourced process to collect

   and reviewed prompts. We do not want to train on

   flooding/toxic/spam/junk/personal information data. We will have a

   leaderboard to motivate the community that shows progress and the most active

   users. Swag will be given to the top-contributors.

2. For each of the collected prompts we will sample multiple completions.

   Completions of one prompt will then be shown randomly to users to rank them

   from best to worst. Again this should happen crowd-sourced, e.g. we need to

   deal with unreliable potentially malicious users. At least multiple votes by

   independent users have to be collected to measure the overall agreement. The

   gathered ranking-data will be used to train a reward model.

3. Now follows the RLHF training phase based on the prompts and the reward

   model.



We can then take the resulting model and continue with completion sampling step

2 for a next iteration.



### Slide Decks



[Vision & Roadmap](https://docs.google.com/presentation/d/1n7IrAOVOqwdYgiYrXc8Sj0He8krn5MVZO_iLkCjTtu0/edit?usp=sharing)



[Important Data Structures](https://docs.google.com/presentation/d/1iaX_nxasVWlvPiSNs0cllR9L_1neZq0RJxd6MFEalUY/edit?usp=sharing)



## How You Can Help



All open source projects begin with people like you. Open source is the belief

that if we collaborate we can together gift our knowledge and technology to the

world for the benefit of humanity.



Check out our [contributing guide](CONTRIBUTING.md) to get started.