{"id":14964724,"url":"https://github.com/ventz/easy-llms","last_synced_at":"2025-03-15T17:06:37.579Z","repository":{"id":242812501,"uuid":"810630464","full_name":"ventz/easy-llms","owner":"ventz","description":"Easy \"1-line\" calling of all LLMs from OpenAI, MS Azure, AWS Bedrock, GCP Vertex, and Ollama","archived":false,"fork":false,"pushed_at":"2024-09-13T16:03:02.000Z","size":74,"stargazers_count":51,"open_issues_count":1,"forks_count":7,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-03-07T03:32:52.634Z","etag":null,"topics":["ai21","amazon-titan","aws","azure","azure-openai","bedrock","claude","claude-3","cohere","gemini-pro","google","google-vertex-ai","langchain","langchain-python","llama3","llms","mistral","mixtral","ollama","openai"],"latest_commit_sha":null,"homepage":"https://pypi.org/project/easy-llms/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ventz.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-06-05T04:24:42.000Z","updated_at":"2024-12-16T16:36:18.000Z","dependencies_parsed_at":"2024-09-24T13:57:57.387Z","dependency_job_id":"d4adf0c5-71de-4bec-9aed-732313105e8f","html_url":"https://github.com/ventz/easy-llms","commit_stats":{"total_commits":14,"total_committers":1,"mean_commits":14.0,"dds":0.0,"last_synced_commit":"934b63327681eaaf55f91e138cc6046b43b159ec"},"previous_names":["ventz/easy-llms"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ventz%2Feasy-llms","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ventz%2Feasy-llms/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ventz%2Feasy-llms/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ventz%2Feasy-llms/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ventz","download_url":"https://codeload.github.com/ventz/easy-llms/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243762268,"owners_count":20343979,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai21","amazon-titan","aws","azure","azure-openai","bedrock","claude","claude-3","cohere","gemini-pro","google","google-vertex-ai","langchain","langchain-python","llama3","llms","mistral","mixtral","ollama","openai"],"created_at":"2024-09-24T13:33:41.641Z","updated_at":"2025-03-15T17:06:37.541Z","avatar_url":"https://github.com/ventz.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"## Easy-LLMS\n* Easy \"1-line\" calling of every LLM from OpenAI, MS Azure, AWS Bedrock, GCP Vertex, and Ollama\n\n## Table of Contents\n- [1.) What is Easy-LLMS and why would you use it](#1-what-is-easy-llms-and-why-would-you-use-it)\n- [2.) Quick Getting Started](#2-quick-getting-started)\n- [3.) Authentication](#3-authentication)\n  - [OpenAI](#openai)\n  - [Azure](#azure)\n  - [AWS](#aws)\n  - [Google](#google)\n  - [Ollama](#ollama)\n  - [Per Model Authentication](#per-model-authentication)\n- [4.) Detailed Usage and Examples](#4-detailed-usage-and-examples)\n  - [Easy \"Import All\" and call any Model](#easy-import-all-and-call-any-model)\n  - [Passing Advanced Options](#passing-advanced-options)\n  - [Loading Providers and Models more efficiently](#loading-providers-and-models-more-efficiently)\n- [5.) Supported Providers and Models](#5-supported-providers-and-models)\n- [6.) Assumptions](#6-assumptions)\n- [7.) Help/Questions/Comments](#7-help-questions-comments)\n\n\n## 1.) What is Easy-LLMS and why would you use it?\nEasy-LLMS is a Python module which gives you easy \"1-line\" access to *every* LLM (Large Language Model) within OpenAI, MS Azure, AWS Bedrock, GCP Vertex, and Ollama\n\nThe idea is that you do not need to focus on the individual provider's APIs or LangChain abstractions, the different authentication methods, or the multitude of LLM requirements/options/parameters/documentation, etc. Rather, you can quickly interact and compare LLMs, get results, and build useful things with those LLMs.\n\nIf you find this helpful, I would very much appreciate starring it on GitHub.\n\n\n\n## 2.) Quick Getting Started\n\n### Install/Upgrade\n* Install/upgrade latest via: `pip install -U easy-llms`\n\n* Alternatively, install latest via: `pip install git+https://github.com/ventz/easy-llms`\n* You can upgrade to the latest via: `pip install --upgrade --force-reinstall git+https://github.com/ventz/easy-llms`\n\n\n### Example: OpenAI\nLet's start with OpenAI since most are familiar with it:\n\n```\nfrom llms.openai import *\n\nanswer = gpt_4o().run(\"what llm are you?\")\n```\n\nThe authentication, in this case for OpenAI, is done in one of two\nways:\n\n1.) Either by having the `OPENAI_API_KEY` ENV variable available in your environment\n\nor\n\n2.) Via having a `.llms` folder in the working directory, with a `openai` file inside of it, which contains `OPENAI_API_KEY=\"sk-...\"`.\n\n\n### Example: AWS Bedrock - Claude v3 Haiku\nNext, let's switch to Claude v3 Haiku via AWS Bedrock.:\n\n\n```\nfrom llms.aws import *\n\nanswer = claude_3_haiku().run(\"what llm are you?\")\n```\n\nThe authentication, in this case for AWS Bedrock, is done in one of two\nways:\n\n1.) Either by having the `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY, and `AWS_REGION` ENV variables available in your environment\n\nor\n\n2.) Via having a `.llms` folder in the working directory, with an `aws` file inside of it, which contains `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY, and `AWS_REGION`.\n\n\n### Example: Ollama - Llama3\nNext, let's switch to Llama3 via Ollama:\n\nOllama does not export any LLMs. This is because it is technically a pass-through to any models that you have setup. \n\nThis means you will *NOT* see it list any LLMs explicitly:\n```\nfrom llms import *\nollama.list()\n```\n\nWhat you would do in this case is pass the model directly to the\nmodule, for example:\n\n```\nfrom llms import *\n\nanswer = ollama(\"llama3\").run(\"what llm are you?\")\n\nBy default, Ollama will use the \"Chat\" interface. If you have a model\nthat does not support Chat, you can disable chat:\n\nanswer = ollama(\"llama3\", chat=False).run(\"what llm are you?\")\n```\n\nWhile Ollama has no authentication, we use the \"BASE_URL\" instead and it is still done in one of two\nways:\n\n1.) Either by having the `OLLAMA_BASE_URL` ENV variable available in your environment\n\nor\n\n2.) Via having a `.llms` folder in the working directory, with a `ollama` file inside of it, which contains `OLLAMA_BASE_URL=\"http://fqdn:11434\"`\n\n\nSee [Authentication](#3-authentication) for more authentication information and examples.\n\nSee [Detailed Usage and Examples](#4-detailed-usage-and-examples) for more usage and options (including advanced parameters) options.\n\n\n\n## 3.) Authentication\n\nThere are two ways of authenticating the various providers within Easy-LLMS:\n\n1.) Via ENV variables\n\nor\n\n2.) Via `.llms` folder in the working directory, with a `openai` file, `azure` file, `aws` file, and `google` file, inside the llms directory, as needed.\n\n\n\nHere are the AUTH requirements for each of the providers in ENV\nvariables and files (choose one, not both)\n\n### OpenAI\n\n1.) ENV variables:\n\n`export OPENAI_API_KEY=\"sk-...\"`\n\nor\n\n2.) `openai` file with:\n```\nOPENAI_API_KEY=\"sk-...\"\n```\n\nNOTE: There is one more variable available: `OPENAI_BASE_URL`, and it\ncan be used to point at an OpenAI compatible proxy. (ex: LiteLLM)\n\nIt can be supplied as an ENV variable, or within the `openai` file.\n\n\n### Azure\n\n1.) ENV variables:\n\n`export AZURE_OPENAI_API_KEY=...`\n\n`export AZURE_OPENAI_API_VERSION=\"YYYY-MM-DD\"`\n\n`export AZURE_OPENAI_ENDPOINT=\"https://your-endpoint.openai.azure.com\"`\n\nor\n\n2.) `azure` file with:\n```\nAZURE_OPENAI_API_KEY=...\nAZURE_OPENAI_API_VERSION=\"YYYY-MM-DD\"\nAZURE_OPENAI_ENDPOINT=\"https://your-endpoint.openai.azure.com\"\n```\n\n### AWS\n\n1.) ENV variables:\n\n`export AWS_ACCESS_KEY_ID=\"AKIA...\"`\n\n`export AWS_SECRET_ACCESS_KEY=\"...\"`\n\n`export AWS_REGION=\"us-east-1\"`\n\nor\n\n2.) `aws` file with:\n```\nAWS_ACCESS_KEY_ID=\"AKIA...\"\nAWS_SECRET_ACCESS_KEY=\"...\"\nAWS_REGION=\"us-east-1\"\n```\n\n### Google\n\n1.) ENV variables:\n\n`export GCP_PROJECT_ID=\"project-name\"`\n\n`export GCP_REGION=\"us-central1\"`\n\n`export GCP_CREDENTIALS_JSON='{\n\"type\": \"service_account\",\n  \"project_id\": \"project-name\",\n  \"private_key_id\": \"...\",\n  \"private_key\": \"-----BEGIN PRIVATE KEY-----\\n...=\\n-----END PRIVATE KEY-----\\n\",\n  \"client_email\": \"project@project-name.iam.gserviceaccount.com\",\n  \"client_id\": \"...\",\n  \"auth_uri\": \"https://accounts.google.com/o/oauth2/auth\",\n  \"token_uri\": \"https://oauth2.googleapis.com/token\",\n  \"auth_provider_x509_cert_url\": \"https://www.googleapis.com/oauth2/v1/certs\",\n  \"client_x509_cert_url\": \"https://www.googleapis.com/robot/v1/metadata/x509/project%40project-name.iam.gserviceaccount.com\",\n  \"universe_domain\": \"googleapis.com\"\n}'`\n\n\nor\n\n2.) `google` file with:\n```\nGCP_PROJECT_ID=\"project-name\"\nGCP_REGION=\"us-central1\"\nGCP_CREDENTIALS_JSON='{\n\"type\": \"service_account\",\n  \"project_id\": \"project-name\",\n  \"private_key_id\": \"...\",\n  \"private_key\": \"-----BEGIN PRIVATE KEY-----\\n...=\\n-----END PRIVATE KEY-----\\n\",\n  \"client_email\": \"project@project-name.iam.gserviceaccount.com\",\n  \"client_id\": \"...\",\n  \"auth_uri\": \"https://accounts.google.com/o/oauth2/auth\",\n  \"token_uri\": \"https://oauth2.googleapis.com/token\",\n  \"auth_provider_x509_cert_url\": \"https://www.googleapis.com/oauth2/v1/certs\",\n  \"client_x509_cert_url\": \"https://www.googleapis.com/robot/v1/metadata/x509/project%40project-name.iam.gserviceaccount.com\",\n  \"universe_domain\": \"googleapis.com\"\n}'\n```\n\n\n### Ollama\n\n1.) ENV variables:\n\n`export OLLAMA_BASE_URL=\"http://fqdn:11434\"`\n\nor\n\n2.) `ollama` file with:\n```\nOLLAMA_BASE_URL=\"http://fqdn:11434\"\n```\n\n## Per Model Authentication\n\nYou can override the auth for a specific model by using the `.llms/provider-model` structure, where `provider` is one of: `openai`, `azure`, `google`, and `ollama`, and model is any of the\nmodels via `provider.list()`\n\nFor example, to override the auth for \"Claude 3 Opus\", since the model\nname is `claude_3_opus` and it's from `aws`, you can create a `.llm/aws-claude_3_opus` config.\n\nOr, to override the auth for \"Titan Premier\", since the model name is `titan_premier_v1` and it's from `aws`, you can create a `.llm/aws-titan_premier_v1` config.\n\nOther than being able to authenticate a specific LLM with different credentials, this is especially useful for certain models that are only available in specific regions (ex: Claude 3 Opus in \"us-west-2\", and Titan Premier in \"us-east-1\")\n\nFor Ollama, again it is worth noting that since it does not export any LLMs, in order to auth a specific model to a specific Ollama instance, you need to pass the `model_auth=True` parameter manually, for example:\n\n```\nfrom llms import *\n\nquestion = \"Who was the first person to step on the moon?\"\n\nanswer = ollama(\"llama3\", model_auth=True).run(question)\n```\n\nThis will look for a `.llms/ollama-llama3` config.\n\n## 4.) Detailed Usage and Examples\n\n1.) Look at [Supported Providers and Models](#5-supported-providers-and-models) to get a list of Providers and Models - a one time task!\n\n2.) Then make sure you have setup the proper [Authentication](#3-authentication) - a one time task!\n\n3.) The easiest and quickest way to use everything:\n\n### Easy \"Import All\" and call any Model\n```\nfrom llms import *\n\n```\n\nFrom here, you can call any provider/model available to you:\n\n```\nanswer = gpt_4o().run(\"what llm are you?\")\n```\n\nor\n\n```\nanswer = claude_3_haiku().run(\"what llm are you?\")\n```\n\nFor example, you can run one question against multiple providers like\nthis:\n\n```\nfrom llms import *\n\nquestion = \"Who was the first person to step on the moon?\"\n\n# OpenAI\nanswer = gpt_4o().run(question)\n\n# AWS\nanswer = claude_3_haiku().run(question)\n\n# Google\nanswer = gemini_pro_1_5().run(question)\n```\n\nIt is worth noting again for that Ollama, you will specify the model\ndirectly, such as:\n\n```\nfrom llms import *\n\nquestion = \"Who was the first person to step on the moon?\"\n\nanswer = ollama(\"llama3\").run(question)\n\nanswer = ollama(\"mistral\").run(question)\n```\n\nAnd again, if your Ollama model does not support \"Chat\", you can specify it without Chat:\n\n```\nfrom llms import *\n\nquestion = \"Who was the first person to step on the moon?\"\n\nanswer = ollama(\"command-r-plus\", chat=False).run(question)\n```\n\n\n\n\n\n### Passing Advanced Options\n\nYou can pass *every* option/parameter that each provider and model supports.\n\nTwo widely used options/parameters are \"wrapped\" for you as:\n\n* `temperature`\nand\n* `max_tokens`\n\nFor example:\n```\nanswer = gpt_4o(temperature=1, max_tokens=100).run(\"what llm are you?\")\n```\n\nYou can pass any other option available, for example:\n\n```\nanswer = gpt_4o(top_p=1).run(\"what llm are you?\")\n```\n\nAnd:\n\n```\nanswer = gpt_4o(top_p=1, presence_penalty=1, frequency_penalty=1).run(\"what llm are you?\")\n```\n\nFor Ollama, it's as simple as:\n\n```\nanswer = ollama(\"llama3\", top_p=1, presence_penalty=1, frequency_penalty=1).run(\"what llm are you?\")\n```\n\n\n\n\n### Loading Providers and Models more efficiently\n\nIf you are only working with one provider, you can only load that\nprovider. For example, you only need OpenAI, you can:\n\n```\nfrom llms.openai import *\nanswer = gpt_4o().run(...)\n```\n\nYou can do the same with Azure:\n\n```\nfrom llms.azure import *\n```\n\nor AWS:\n\n```\nfrom llms.aws import *\n```\n\nor Google:\n\n```\nfrom llms.google import *\n```\n\n\nYou can further narrow it down to one model. For example, let's say\nyou only need \"gpt-4o\" from OpenAI, you can load it as:\n```\nfrom llms.openai import gpt_4o\nanswer = gpt_4o().run(...)\n```\n\n\nAgain, Ollama is the only one that you *CANNOT* load with:\n```\nfrom llms.ollama import *\nanswer = ollama(\"llama3\").run(...)\n```\n\nThis is due to Ollama not exporting any LLMs.\n\nYou would have load it with:\n```\nfrom llms import ollama\nanswer = ollama(\"llama3\").run(...)\n```\n\n\n\n## 5.) Supported Providers and Models\n\nYou can list out the models with:\n\n```\nfrom llms import llms\nprint(llms.list())\n```\n\nWhich will produce:\n```\n['openai', 'azure', 'aws', 'google', 'ollama']\n```\n\nAnd from there, you can list each as:\n\n```\nfrom llms import openai\nprint(openai.list())\n\nfrom llms import azure\nprint(azure.list())\n\nfrom llms import aws\nprint(aws.list())\n\nfrom llms import google\nprint(google.list())\n\n# NOTE: Ollama does not export any LLMs.\n#       This is because it is technically a pass-through to any models that you have setup. \n#       That means you will see an empty list \"[]\" of exported LLMs \nfrom llms import ollama\nprint(ollama.list())\n\n```\n\nWhich will produce the models you can use:\n```\n['gpt_35_turbo', 'gpt_35_turbo_16k', 'gpt_4_turbo', 'gpt_4o', 'gpt_4o_mini', 'gpt_4', 'gpt_4_32k']\n\n['azure_gpt_35_turbo', 'azure_gpt_35_turbo_16k', 'azure_gpt_4_turbo', 'azure_gpt_4', 'azure_gpt_4_32k']\n\n['claude_3_haiku', 'claude_3_sonnet', 'claude_3_opus', 'claude_35_sonnet', 'claude_1_instant', 'claude_1', 'llama2_70b', 'llama3_8b_instruct', 'llama3_70b_instruct', 'llama3_1_8b_instruct', 'llama3_1_70b_instruct', 'llama3_1_405b_instruct', 'mistral_7b_instruct', 'mistral_large', 'mistral_large_2', 'mistral_small', 'mixtral_8x7b_instruct', 'cohere_command_14', 'cohere_command_light_14', 'j2_mid', 'j2_ultra', 'titan_lite_v1', 'titan_express_v1', 'titan_premier_v1']\n\n['gemini_pro_1', 'gemini_pro_1_5', 'gemini_flash_1_5', 'bison', 'bison_32k']\n\n# NOTE: Ollama is empty due to not exporting any LLMs\n[]\n```\n\n\n\nHere is a user-friendly list of the models across OpenAI, Azure, AWS, Google, and Ollama, however do note that these are NOT the model names that you would use within Easy-LLMS:\n\n```\n# AWS Anthropic:\n* claude-3-haiku\n* claude-3-sonnet\n* claude_35_sonnet\n* claude-3-opus [note: currently only available in 'us-east-2']\n* claude-v2\n* claude-instant\n\n# AWS Meta:\n* llama2-13b\n* llama2-70b\n* llama3-8b\n* llama3-70b\n* llama3.1-8b\n* llama3.1-70b\n* llama3.1-405b\n\n# AWS Mistral and Mixtral:\n* mistral-7b-instruct\n* mistral-large\n* mistral-large-2\n* mistral-small\n* mixtral-8x7b-instruct\n* mixtral-large\n\n# AWS Cohere:\n* cohere-command-v14\n* cohere-command-light-v14\n* [support coming soon] cohere.command-r-v1:0\n* [support coming soon] cohere.command-r-plus-v1:0\n\n# AWS AI21 Labs:\n* ai21-j2-mid\n* ai21-j2-ultra\n\n# AWS Amazon (Titan):\n* amazon-titan-lite\n* amazon-titan-express\n* amazon-titan-premier [note: currently only available in 'us-east-1']\n\n# MS Azure:\n* azure-gpt-4-turbo-preview\n* azure-gpt-3.5-turbo\n* azure-gpt-4o\n* azure-gpt-4\n* azure-gpt-3.5-turbo-16k\n* azure-gpt-4-32k\n\n# OpenAI:\n* gpt-4-turbo\n* gpt-4-turbo-preview\n* gpt-3.5-turbo\n* gpt-4o\n* gpt-4o-mini\n* gpt-4\n* gpt-3.5-turbo-16k\n* gpt-4-32k\n\n# Google Vertex:\n* google-chat-bison\n* google-chat-bison-32k\n* google-gemini-pro-1.0\n* google-gemini-pro-1.5-preview [note: currently only available in 'us-central1']\n* google-gemini-flash-1.5-preview [note: currently only available in 'us-central1']\n\n# Ollama\n# https://ollama.com/library\n* Any \"Chat\" model\n* Any \"Invoke\" model\n```\n\n\n## 6.) Assumptions?\n\n* You have python3 (ideally 3.11.x+, although it should work 3.6.x+)\n\n* You have a python venv setup\n\n* You have `pip install wheel` in your venv\n\n* You have installed Easy-LLMS: `pip install easy-llms` or `pip install git+https://github.com/ventz/easy-llms`\n\n* You have API access to the providers/models you want to use: OpenAI; MS Azure; AWS Bedrock; Google Vertex; Ollama\nSee [Supported Providers and Models](#supported-providers-and-models) for more information.\n\n\n## 7.) Help/Questions/Comments\nPlease feel free to open GitHub Issues for all Questions and Comments.\nPRs are always welcome and encouraged! The goal is to make this project better!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fventz%2Feasy-llms","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fventz%2Feasy-llms","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fventz%2Feasy-llms/lists"}