{"id":14964909,"url":"https://github.com/datastax/astra-assistants-api","last_synced_at":"2025-04-15T03:50:44.750Z","repository":{"id":209446899,"uuid":"718954997","full_name":"datastax/astra-assistants-api","owner":"datastax","description":"Drop in replacement for the OpenAI Assistants API ","archived":false,"fork":false,"pushed_at":"2025-03-17T15:01:16.000Z","size":3991,"stargazers_count":187,"open_issues_count":18,"forks_count":22,"subscribers_count":10,"default_branch":"main","last_synced_at":"2025-04-07T17:04:37.771Z","etag":null,"topics":["assistants","assistants-api","claude","cohere","gemini","gpt-4","groq","llama3-1","ollama","openai-assistants","openai-assistants-api","vector"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/datastax.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-11-15T06:10:19.000Z","updated_at":"2025-03-15T00:35:34.000Z","dependencies_parsed_at":"2023-12-22T12:52:08.453Z","dependency_job_id":"8936299e-36d1-475f-bdc7-f11a1b367b17","html_url":"https://github.com/datastax/astra-assistants-api","commit_stats":{"total_commits":469,"total_committers":4,"mean_commits":117.25,"dds":0.02558635394456288,"last_synced_commit":"2af11709a1e5e199c4b2e0df8e4a872a078579a5"},"previous_names":["datastax/astra-assistants-api"],"tags_count":93,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datastax%2Fastra-assistants-api","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datastax%2Fastra-assistants-api/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datastax%2Fastra-assistants-api/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datastax%2Fastra-assistants-api/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/datastax","download_url":"https://codeload.github.com/datastax/astra-assistants-api/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249003944,"owners_count":21196794,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["assistants","assistants-api","claude","cohere","gemini","gpt-4","groq","llama3-1","ollama","openai-assistants","openai-assistants-api","vector"],"created_at":"2024-09-24T13:33:57.412Z","updated_at":"2025-04-15T03:50:44.728Z","avatar_url":"https://github.com/datastax.png","language":"Python","funding_links":[],"categories":["APIs","Learning","Python","LLM Tools"],"sub_categories":["Repositories"],"readme":"# Astra Assistant API Service\n[![commits](https://img.shields.io/github/commit-activity/m/datastax/astra-assistants-api)](https://github.com/datastax/astra-assistants-api/commits/main)\n[![Github Last Commit](https://img.shields.io/github/last-commit/datastax/astra-assistants-api)](https://github.com/datastax/astra-assistants-api/commits/main)\n[![Run tests](https://github.com/datastax/astra-assistants-api/actions/workflows/run-tests.yml/badge.svg?branch=main)](https://github.com/datastax/astra-assistants-api/actions/workflows/run-tests.yml)\n[![Docker build and publish](https://github.com/datastax/astra-assistants-api/actions/workflows/docker.yml/badge.svg)](https://github.com/datastax/astra-assistants-api/actions/workflows/docker.yml)\n[![PyPI - Downloads](https://img.shields.io/pypi/dw/astra-assistants?label=pypi%20downloads)](https://badge.fury.io/py/astra-assistants)\n[![Dockerhub](https://img.shields.io/static/v1?label=Pull%20from\u0026message=DockerHub\u0026color=blue\u0026logo=Docker\u0026style=flat-square)](https://hub.docker.com/r/datastax/astra-assistants)\n[![Discord chat](https://img.shields.io/static/v1?label=Chat%20on\u0026message=Discord\u0026color=blue\u0026logo=Discord\u0026style=flat-square)](https://discord.gg/MEFVXUvsuy)\n[![Stars](https://img.shields.io/github/stars/datastax/astra-assistants-api?style=social)](https://github.com/datastax/astra-assistants-api/stargazers)\n\n### An Open Source drop-in compatible service for the OpenAI Assistants API v2\n\n![create_assistant](images/create_assistant.gif)\n\n\nAstra Assistants supports streaming, persistent threads, files, vector_stores, assistants, retrieval, function calling and more using [AstraDB](https://astra.datastax.com) (DataStax's db as a service offering powered by [Apache Cassandra](https://cassandra.apache.org/_/index.html) and [jvector](https://github.com/jbellis/jvector)).\n\nSupports dozens of third party LLM providers (or even local models) for both completion and embeddings (powered by [LiteLLM](https://github.com/BerriAI/litellm)). \n\nYou can use our hosted Astra Assistants service, or host the open source API server yourself.\n\n## Client Getting Started [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/gist/phact/a80dc113dd637ba4c4193415e69198c6/assistants_api_overview_python.ipynb)\n\nTo build an app that uses the Astra Asistants service install the [astra-assistants](https://pypi.org/project/astra-assistants/) python library with your favorite package manager. The code for astra-assistants can be found under (clients/)[./clients/]:\n\n```\npoetry add astra_assistants\n```\n\n[Signup for Astra and get an Admin API token](https://astra.datastax.com/signup):\n\nSet your environment variables (depending on what LLMs you want to use), see the [.env.bkp](./.env.bkp) file for an example:\n\n```\n#!/bin/bash\n\n# AstraDB -\u003e https://astra.datastax.com/ --\u003e tokens --\u003e administrator user --\u003e generate\nexport ASTRA_DB_APPLICATION_TOKEN=\"\"\n\n# OpenAI Models - https://platform.openai.com/api-keys --\u003e create new secret key\nexport OPENAI_API_KEY=\"\"\n\n# Groq Models - https://console.groq.com/keys\nexport GROQ_API_KEY=\"\"\n\n# Anthropic claude models - https://console.anthropic.com/settings/keys\nexport ANTHROPIC_API_KEY=\"\"\n\n# Gemini models -\u003e https://makersuite.google.com/app/apikey\nexport GEMINI_API_KEY=\"\"\n\n# Perplexity models -\u003e https://www.perplexity.ai/settings/api  --\u003e generate\nexport PERPLEXITYAI_API_KEY=\"\"\n\n# Cohere models -\u003e https://dashboard.cohere.com/api-keys\nexport COHERE_API_KEY=\"\"\n\n# Bedrock models -\u003e https://docs.aws.amazon.com/bedrock/latest/userguide/setting-up.html\nexport AWS_REGION_NAME=\"\"\nexport AWS_ACCESS_KEY_ID=\"\"\nexport AWS_SECRET_ACCESS_KEY=\"\"\n\n# vertexai models https://console.cloud.google.com/vertex-ai\nexport GOOGLE_JSON_PATH=\"\"\nexport GOOGLE_PROJECT_ID=\"\"\n\n# ... for all models see the .env.bkp file\n```\n\nThen import and patch your client:\n\n```python\nfrom openai import OpenAI\nfrom astra_assistants import patch\nclient = patch(OpenAI())\n```\nThe system will create a db on your behalf and name it `assistant_api_db` using your token. Note, this means that the first request will hang until your db is ready (could be a couple of minutes). This will only happen once.\n\nNow you're ready to create an assistant\n\n```\nassistant = client.beta.assistants.create(\n  instructions=\"You are a personal math tutor. When asked a math question, write and run code to answer the question.\",\n  model=\"gpt-4-1106-preview\",\n  tools=[{\"type\": \"retrieval\"}]\n)\n```\n\nBy default, the service uses [AstraDB](https://astra.datastax.com/signup) as the database/vector store and OpenAI for embeddings and chat completion.\n\n\n## Third party LLM Support\n\nWe now support [many third party models](https://docs.litellm.ai/docs/providers) for both embeddings and completion thanks to [litellm](https://github.com/BerriAI/litellm). Pass the api key of your service using `api-key` and `embedding-model` headers.\n\nYou can pass different models, just make sure you have the right corresponding api key in your environment.\n\n```\nmodel=\"gpt-4-1106-preview\"\n#model=\"gpt-3.5-turbo\"\n#model=\"cohere_chat/command-r\"\n#model=\"perplexity/mixtral-8x7b-instruct\"\n#model=\"perplexity/llama-3-sonar-large-32k-online\"\n#model=\"anthropic.claude-v2\"\n#model=\"gemini/gemini-1.5-pro-latest\"\n#model = \"meta.llama2-13b-chat-v1\"\n\nassistant = client.beta.assistants.create(\n    name=\"Math Tutor\",\n    instructions=\"You are a personal math tutor. Answer questions briefly, in a sentence or less.\",\n    model=model,\n)\n```\n\nfor third party embedding models we support `embedding_model` in `client.files.create`:\n```\nfile = client.files.create(\n    file=open(\n        \"./test/language_models_are_unsupervised_multitask_learners.pdf\",\n        \"rb\",\n    ),\n    purpose=\"assistants\",\n    embedding_model=\"text-embedding-3-large\",\n)\n```\n\nTo run the examples using poetry create a .env file in this directory with your secrets and run:\n\n    poetry install\n\nCreate your .env file and add your keys to it:\n\n    cp .env.bkp .env\n\nand \n\n    poetry run python examples/python/chat_completion/basic.py\n\n    poetry run python examples/python/retrieval/basic.py\n\n    poetry run python examples/python/streaming_retrieval/basic.py\n\n    poetry run python examples/python/function_calling/basic.py\n\n\n## Running yourself\n\n### Docker\n\nwith docker, first pull the image from docker hub\n\n    docker pull datastax/astra-assistants\n\nor a specific version if you don't want latest:\n\n    docker pull datastax/astra-assistants:v0.2.12\n\nthen run (-p to map your docker port 8080 to your host port 8080):\n\n    docker run -p 8080:8080 datastax/astra-assistants\n\n\n### Locally with poetry\n\nor locally with poetry:\n\n    poetry install\n\n    poetry run python run.py\n\n### Docker-compose with ollama\n\nor with docker-compose for integration with ollama\n\n    cd examples/ollama/gpu # or examples/ollama/cpu for cpu only for gpu you need docker-toolkit\n\n    docker-compose up -d\n\nyou need to pull the model you want to ollama before using it\n\n    curl http://localhost:11434/api/pull -d '{ \"name\": \"deepseek-coder-v2\" }'\n\nyour assistants client should route to the ollama container setting OLLAMA_API_BASE_URL. OLLAMA_API_BASE_URL should be set to http://ollama:11434 if you are using docker-compose. If you are using ollama on your localhost you can set it to http://localhost:11434\n\n\n## Feedback / Help\nFor help or feedback file an [issue](https://github.com/datastax/astra-assistants-api/issues) or reach out to us on [Discord](https://discord.com/invite/MEFVXUvsuy)\n\n## Contributing\n\nCheck out our [contributing guide](./CONTRIBUTING.md)\n\n## Coverage\n\nSee our coverage report [here](./coverage.md)\n\n## Roadmap:\n - [X] Support for other embedding models and LLMs\n - [X] function calling\n - [X] Streaming support\n - [X] Assistants V2 with vector_store support\n\n\nThis project is not associated with OpenAI or any of the third party models we support. It is an open source project that aims to provide a drop-in compatible service for the OpenAI Assistants API.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatastax%2Fastra-assistants-api","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdatastax%2Fastra-assistants-api","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatastax%2Fastra-assistants-api/lists"}