{"id":24847538,"url":"https://github.com/mimirtechco/dokploy-litellm","last_synced_at":"2025-03-26T10:20:44.132Z","repository":{"id":274535483,"uuid":"923220259","full_name":"mimirtechco/dokploy-litellm","owner":"mimirtechco","description":"LiteLLM Docker Compose prepared for using on Dokploy","archived":false,"fork":false,"pushed_at":"2025-01-27T21:32:07.000Z","size":8,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-27T22:32:48.872Z","etag":null,"topics":["docker-compose","dokploy","litellm"],"latest_commit_sha":null,"homepage":"https://www.mimirtech.co","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mimirtechco.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-01-27T20:55:52.000Z","updated_at":"2025-01-27T21:32:11.000Z","dependencies_parsed_at":"2025-01-27T22:32:52.191Z","dependency_job_id":"33fd62da-d0ea-49a7-a641-969cff9de873","html_url":"https://github.com/mimirtechco/dokploy-litellm","commit_stats":null,"previous_names":["mimirtechco/dokploy-litellm"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mimirtechco%2Fdokploy-litellm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mimirtechco%2Fdokploy-litellm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mimirtechco%2Fdokploy-litellm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mimirtechco%2Fdokploy-litellm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mimirtechco","download_url":"https://codeload.github.com/mimirtechco/dokploy-litellm/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245632351,"owners_count":20647194,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["docker-compose","dokploy","litellm"],"created_at":"2025-01-31T11:29:55.113Z","updated_at":"2025-03-26T10:20:44.106Z","avatar_url":"https://github.com/mimirtechco.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# LiteLLM for Dokploy\n\nLiteLLM Docker Compose prepared for using on Dokploy.\n\n## Requirements\n\n### PostgreSQL Service\n\nYou need to start a PostgreSQL service for database.\n\n### Redis Servcie\n\nYou need to start a redis service for cache.\n\n## Envioriments variables\n\n```env\n\nHASH=\u003c\u003e\nPROXY_PORT=4000\nPROXY_HOST=\u003c\u003e\nNUM_WORKERS=8\nLITELLM_MASTER_KEY=\u003c\u003e\nLITELLM_SALT_KEY=\u003c\u003e\nLITELLM_LOG=\"ERROR\"\nLITELLM_MODE=\"PRODUCTION\"\nDATABASE_URL=\u003c\u003e\nREDIS_HOST=\u003c\u003e\nREDIS_PASSWORD=\u003c\u003e\nREDIS_USER=default\nREDIS_PORT=6379\nUI_USERNAME=\u003c\u003e\nUI_PASSWORD=\u003c\u003e\n\nSENTRY_DSN=\u003c\u003e\nSENTRY_TRACES_SAMPLE_RATE=1.0\nSENTRY_ENABLED=true\nLANGFUSE_HOST=\u003c\u003e\nLANGFUSE_PUBLIC_KEY=\u003c\u003e\nLANGFUSE_SECRET_KEY=\u003c\u003e\n\n# Providers API\nOPENAI_API_KEY=\u003copen_api_key\u003e\nGROQ_API_KEY=\u003c\u003e\nCLOUDFLARE_API_KEY=\u003c\u003e\nCLOUDFLARE_API_BASE=https://gateway.ai.cloudflare.com/v1/\u003caccount\u003e/\u003caigateway\u003e/workers-ai/v1\nGITHUB_API_KEY=\u003c\u003e\n\n```\n\n## Config.yaml template\n\n```yaml\nmodel_list:\n# OpenAI\n  - model_name: dall-e-3 # alias for dall-e-3\n    litellm_params:\n      model: openai/dall-e-3\n      api_key: os.environ/OPENAI_API_KEY\n    model_info:\n      mode: image_generation\n      base_model: dall-e-3\n  - model_name: openai-dall-e-3\n    litellm_params:\n      model: openai/dall-e-3\n      api_key: os.environ/OPENAI_API_KEY\n    model_info:\n      mode: image_generation\n      base_model: dall-e-3\n  - model_name: openai-dall-e-2 # alias for dall-e-2\n    litellm_params:\n      model: openai/dall-e-2\n      api_key: os.environ/OPENAI_API_KEY\n    model_info:\n      mode: image_generation\n      base_model: dall-e-2\n  - model_name: openai-dall-e-2\n    litellm_params:\n      model: openai/dall-e-2\n      api_key: os.environ/OPENAI_API_KEY\n    model_info:\n      mode: image_generation\n      base_model: dall-e-2\n  - model_name: gpt-3.5-turbo\n    litellm_params:\n      model: openai/gpt-3.5-turbo   \n      api_key: os.environ/OPENAI_API_KEY\n  - model_name: gpt-3.5-turbo-instruct\n    litellm_params:\n      model: text-completion-openai/gpt-3.5-turbo-instruct\n      api_key: os.environ/OPENAI_API_KEY\n  - model_name: gpt-3.5-turbo-large\n    litellm_params: \n      model: \"gpt-3.5-turbo-1106\"\n      api_key: os.environ/OPENAI_API_KEY\n      rpm: 480\n      timeout: 300\n      stream_timeout: 60\n  - model_name: \"davinci\"\n    litellm_params:\n      model: \"openai/davinci-002\"\n      api_key: os.environ/OPENAI_API_KEY\n\n# Grog\n  - model_name: \"groq/*\"\n    litellm_params:\n      model: \"groq/*\"\n      api_key: os.environ/GROQ_API_KEY\n\n# Cloudflare\n  - model_name: \"phi-2\"\n    litellm_params:\n      model: \"openai/@cf/microsoft/phi-2\"\n      api_key: os.environ/CLOUDFLARE_API_KEY\n      api_base: os.environ/CLOUDFLARE_API_BASE\n  - model_name: \"starling-lm-7b\"\n    litellm_params:\n      model: \"openai/@hf/nexusflow/starling-lm-7b-beta\"\n      api_key: os.environ/CLOUDFLARE_API_KEY\n      api_base: os.environ/CLOUDFLARE_API_BASE\n    model_info:\n      supports_vision: True\n  - model_name: \"mistral-7b\"\n    litellm_params:\n      model: \"openai/@hf/mistral/mistral-7b-instruct-v0.2\"\n      api_key: os.environ/CLOUDFLARE_API_KEY\n      api_base: os.environ/CLOUDFLARE_API_BASE\n  - model_name: \"cf/llama-3.2-11b\"\n    litellm_params:\n      model: \"openai/@cf/meta/llama-3.2-11b-vision-instruct\"\n      api_key: os.environ/CLOUDFLARE_API_KEY\n      api_base: os.environ/CLOUDFLARE_API_BASE\n    model_info:\n      supports_vision: True\n  - model_name: \"llama-3.2-3b\"\n    litellm_params:\n      model: \"openai/@cf/meta/llama-3.2-3b-instruct\"\n      api_key: os.environ/CLOUDFLARE_API_KEY\n      api_base: os.environ/CLOUDFLARE_API_BASE\n      max_tokens: 4096\n  - model_name: \"llama-2-7b-chat-fp16\"\n    litellm_params:\n      model: \"openai/@cf/meta/llama-2-7b-chat-fp16\"\n      api_key: os.environ/CLOUDFLARE_API_KEY\n      api_base: os.environ/CLOUDFLARE_API_BASE    \n\n# Github\n  - model_name: \"llama3-8b\"\n    litellm_params:\n      model: \"github/Meta-Llama-3-8B-Instruct\"\n      api_key: os.environ/GITHUB_API_KEY\n      rpm: 10\n  - model_name: \"llama3.2-11b\"\n    litellm_params:\n      model: \"github/Llama-3.2-11B-Vision-Instruct\"\n      api_key: os.environ/GITHUB_API_KEY\n      rpm: 10\n    model_info:\n      supports_vision: True\n  - model_name: \"mistral-large\"\n    litellm_params:\n      model: \"github/Mistral-large-2407\"\n      api_key: os.environ/GITHUB_API_KEY\n      rpm: 10\n  - model_name: \"mistral-nemo\"\n    litellm_params:\n      model: \"github/Mistral-Nemo\"\n      api_key: os.environ/GITHUB_API_KEY\n      rpm: 10\n  - model_name: \"cohere-plus\"\n    litellm_params:\n      model: \"github/Cohere-command-r-plus-08-2024\"\n      api_key: os.environ/GITHUB_API_KEY\n      rpm: 10\n  - model_name: \"phi-3\"\n    litellm_params:\n      model: \"github/Phi-3-medium-128k-instruct\"\n      api_key: os.environ/GITHUB_API_KEY\n      rpm: 10\n  - model_name: \"phi-3.5-MoE\"\n    litellm_params:\n      model: \"github/Phi-3.5-MoE-instruct\"\n      api_key: os.environ/GITHUB_API_KEY\n      rpm: 10\n  - model_name: \"Phi-4\"\n    litellm_params:\n      model: \"github/Phi-4\"\n      api_key: os.environ/GITHUB_API_KEY\n      rpm: 10\n  - model_name: \"gpt-4o\"\n    litellm_params:\n      model: \"github/gpt-4o\"\n      api_key: os.environ/GITHUB_API_KEY\n      rpm: 10\n    model_info:\n      supports_vision: True\n  - model_name: \"gpt-4o-mini\"\n    litellm_params:\n      model: \"github/gpt-4o-mini\"\n      api_key: os.environ/GITHUB_API_KEY\n      rpm: 10\n    model_info:\n      supports_vision: True\n  - model_name: \"jamba\"\n    litellm_params:\n      model: \"github/AI21-Jamba-1.5-Large\"\n      api_key: os.environ/GITHUB_API_KEY\n      rpm: 10\n  - model_name: \"text-embedding-3-large\"\n    litellm_params:\n      model: \"github/text-embedding-3-large\"\n      api_key: os.environ/GITHUB_API_KEY\n      rpm: 15\n    model_info:\n      mode: embedding\n\nlitellm_settings:\n  set_verbose: False  # Uncomment this if you want to see verbose logs; not recommended in production\n  drop_params: True\n  # max_budget: 100 \n  # budget_duration: 30d\n  num_retries: 5\n  request_timeout: 120\n  telemetry: True\n  set_verbose: True\n  cache: True\n  #callbacks: [\"langfuse\"]\n  redact_user_api_key_info: true\n  input_callback: [\"langfuse\"]\n  failure_callback: [\"sentry\"]\n  success_callback: [\"langfuse\"]\n#  context_window_fallbacks: [{\"llama3-8b\": [\"llama3-8b-8192\"]}]\n\nrouter_settings:\n  routing_strategy: usage-based-routing-v2 \n  redis_host: os.environ/REDIS_HOST\n  redis_password: os.environ/REDIS_PASSWORD\n  redis_port: os.environ/REDIS_PORT\n  timeout: 120\n#  enable_pre_call_checks: true\n#  model_group_alias: {\"my-special-fake-model-alias-name\": \"fake-openai-endpoint-3\"} \n\ngeneral_settings: \n  master_key: os.environ/LITELLM_MASTER_KEY # [OPTIONAL] Use to enforce auth on proxy. See - https://docs.litellm.ai/docs/proxy/virtual_keys\n  store_model_in_db: False\n  disable_spend_logs: False\n  #proxy_budget_rescheduler_min_time: 60\n  #proxy_budget_rescheduler_max_time: 65\n  proxy_batch_write_at: 60\n  database_connection_pool_limit: 20\n  database_connection_timeout: 45\n  database_url: os.environ/DATABASE_URL # [OPTIONAL] use for token-based auth to proxy\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmimirtechco%2Fdokploy-litellm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmimirtechco%2Fdokploy-litellm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmimirtechco%2Fdokploy-litellm/lists"}