{"id":15374600,"url":"https://github.com/ading2210/vercel-llm-api","last_synced_at":"2025-04-09T14:06:43.697Z","repository":{"id":170643866,"uuid":"644820185","full_name":"ading2210/vercel-llm-api","owner":"ading2210","description":"A reverse engineered Python API wrapper for the Vercel AI Playground, which provides free access to many large language models without needing an account.","archived":false,"fork":false,"pushed_at":"2023-10-27T15:48:44.000Z","size":36,"stargazers_count":175,"open_issues_count":6,"forks_count":13,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-04-02T13:06:33.786Z","etag":null,"topics":["chatgpt","library","openai","python","vercel"],"latest_commit_sha":null,"homepage":"https://pypi.org/project/vercel-llm-api","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ading2210.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-05-24T10:21:36.000Z","updated_at":"2025-03-19T22:55:34.000Z","dependencies_parsed_at":null,"dependency_job_id":"96adb7e6-ac2a-4d32-966a-6093c3f56ae2","html_url":"https://github.com/ading2210/vercel-llm-api","commit_stats":{"total_commits":18,"total_committers":1,"mean_commits":18.0,"dds":0.0,"last_synced_commit":"e75433787dbf590d244ff454efb553b4f14ab06c"},"previous_names":["ading2210/vercel-llm-api"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ading2210%2Fvercel-llm-api","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ading2210%2Fvercel-llm-api/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ading2210%2Fvercel-llm-api/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ading2210%2Fvercel-llm-api/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ading2210","download_url":"https://codeload.github.com/ading2210/vercel-llm-api/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248054227,"owners_count":21039952,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chatgpt","library","openai","python","vercel"],"created_at":"2024-10-01T13:59:08.200Z","updated_at":"2025-04-09T14:06:43.657Z","avatar_url":"https://github.com/ading2210.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Python Vercel LLM API\n\n[![PyPi Version](https://img.shields.io/pypi/v/vercel-llm-api.svg)](https://pypi.org/project/vercel-llm-api/)\n\nThis is a reverse engineered API wrapper for the [Vercel AI Playground](https://play.vercel.ai/), which allows for free access to many LLMs, including OpenAI's ChatGPT, Cohere's Command Nightly, as well as some open source models.\n\n## Table of Contents:\n- [Features](#features)\n- [Limitations](#limitations)\n- [Installation](#installation)\n- [Documentation](#documentation)\n  * [Using the Client](#using-the-client)\n    + [Downloading the Available Models](#downloading-the-available-models)\n    + [Generating Text](#generating-text)\n    + [Generating Chat Messages](#generating-chat-messages)\n  * [Misc](#misc)\n    + [Changing the Logging Level](#changing-the-logging-level)\n- [Copyright](#copyright)\n  * [Copyright Notice](#copyright-notice)\n\n*Table of contents generated with [markdown-toc](http://ecotrust-canada.github.io/markdown-toc).*\n\n## Features:\n - Download the available models\n - Generate text\n - Generate chat messages\n - Set custom parameters\n - Stream the responses\n\n## Limitations:\n - User-agent is hardcoded\n - No auth support\n - Can't use \"pro\" or \"hobby\" models\n\n## Installation:\nYou can install this library by running the following command:\n```\npip3 install vercel-llm-api\n```\n\n## Documentation:\nExamples can be found in the `/examples` directory. To run these examples, simply execute the included Python files from your terminal.\n```\npython3 examples/generate.py\n```\n\n### Using the Client:\nTo use this library, simply import `vercel_ai` and create a `vercel_ai.Client` instance. You can specify a proxy using the `proxy` keyword argument.\n\nNormal example:\n```python\nimport vercel_ai\nclient = vercel_ai.Client()\n```\n\nProxied example:\n```python\nimport vercel_ai\nclient = vercel_ai.Client(proxy=\"socks5h://193.29.62.48:11003\")\n```\n\nNote that the following examples assume `client` is the name of your `vercel_ai.Client` instance.\n\n#### Downloading the Available Models:\nThe client downloads the available models upon initialization, and stores them in `client.models`. \n```python\n\u003e\u003e\u003e print(json.dumps(client.models, indent=2))\n\n{\n  \"anthropic:claude-instant-v1\": { \n    \"id\": \"anthropic:claude-instant-v1\", #the model's id\n    \"provider\": \"anthropic\",             #the model's provider\n    \"providerHumanName\": \"Anthropic\",    #the provider's display name\n    \"makerHumanName\": \"Anthropic\",       #the maker of the model\n    \"minBillingTier\": \"hobby\",           #the minimum billing tier needed to use the model\n    \"parameters\": {                      #a dict of optional parameters that can be passed to the generate function\n      \"temperature\": {                   #the name of the parameter\n        \"value\": 1,                      #the default value for the parameter\n        \"range\": [0, 1]                  #a range of possible values for the parameter\n      },\n      ...\n    }\n    ...\n  }\n}\n```\nNote that, since there is no auth yet, if a model has the `\"minBillingTier\"` property present, it can't be used.\n\nA list of model IDs is also available in `client.model_ids`.\n```python\n\u003e\u003e\u003e print(json.dumps(client.model_ids, indent=2))\n[\n  \"anthropic:claude-instant-v1\", #locked to hobby tier; unusable\n  \"anthropic:claude-v1\",         #locked to hobby tier; unusable\n  \"replicate:replicate/alpaca-7b\",\n  \"replicate:stability-ai/stablelm-tuned-alpha-7b\",\n  \"huggingface:bigscience/bloom\",\n  \"huggingface:bigscience/bloomz\",\n  \"huggingface:google/flan-t5-xxl\",\n  \"huggingface:google/flan-ul2\",\n  \"huggingface:EleutherAI/gpt-neox-20b\",\n  \"huggingface:OpenAssistant/oasst-sft-4-pythia-12b-epoch-3.5\",\n  \"huggingface:bigcode/santacoder\",\n  \"cohere:command-medium-nightly\",\n  \"cohere:command-xlarge-nightly\",\n  \"openai:gpt-4\",                #locked to pro tier; unusable\n  \"openai:code-cushman-001\",\n  \"openai:code-davinci-002\",\n  \"openai:gpt-3.5-turbo\",\n  \"openai:text-ada-001\",\n  \"openai:text-babbage-001\",\n  \"openai:text-curie-001\",\n  \"openai:text-davinci-002\",\n  \"openai:text-davinci-003\"\n]\n```\n\nA dict of default parameters for each model can be found at `client.model_params`.\n```python\n\u003e\u003e\u003e print(json.dumps(client.model_defaults, indent=2))\n{\n  \"anthropic:claude-instant-v1\": {\n    \"temperature\": 1,\n    \"maximumLength\": 200,\n    \"topP\": 1,\n    \"topK\": 1,\n    \"presencePenalty\": 1,\n    \"frequencyPenalty\": 1,\n    \"stopSequences\": [\n      \"\\n\\nHuman:\"\n    ]\n  },\n  ...\n}\n```\n\n#### Generating Text:\nTo generate some text, use the `client.generate` function, which accepts the following arguments:\n - `model` - The ID of the model you want to use.\n - `prompt` - Your prompt.\n - `params = {}` - A dict of optional parameters. See the previous section for how to find these.\n\nThe function is a generator which returns the newly generated text as a string.\n\nStreamed Example:\n```python\nfor chunk in client.generate(\"openai:gpt-3.5-turbo\", \"Summarize the GNU GPL v3\"):\n  print(chunk, end=\"\", flush=True)\n```\n\nNon-Streamed Example:\n```python\nresult = \"\"\nfor chunk in client.generate(\"openai:gpt-3.5-turbo\", \"Summarize the GNU GPL v3\"):\n  result += chunk\nprint(result)\n```\n\n#### Generating Chat Messages:\nTo generate chat messages, use the `client.chat` function, which accepts the following arguments:\n - `model` - The ID of the model you want to use.\n - `messages` - A list of messages. The format for this is identical to how you would use the official OpenAI API.\n - `params = {}` - A dict of optional parameters. See the \"Downloading the Available Models\" section for how to find these.\n\nThe function is a generator which returns the newly generated text as a string.\n\n```python\nmessages = [\n  {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n  {\"role\": \"user\", \"content\": \"Who won the world series in 2020?\"},\n  {\"role\": \"assistant\", \"content\": \"The Los Angeles Dodgers won the World Series in 2020.\"},\n  {\"role\": \"user\", \"content\": \"Where was it played?\"}\n]\nfor chunk in client.chat(\"openai:gpt-3.5-turbo\", messages):\n  print(chunk, end=\"\", flush=True)\nprint()\n```\n\n### Misc:\n\n#### Changing the Logging Level:\nIf you want to show the debug messages, simply call `vercel_ai.logger.setLevel`.\n\n```python\nimport vercel_ai\nimport logging\nvercel_ai.logger.setLevel(logging.INFO)\n```\n\n## Copyright:\nThis program is licensed under the [GNU GPL v3](https://github.com/ading2210/vercel-llm-api/blob/main/LICENSE). All code has been written by me, [ading2210](https://github.com/ading2210).\n\n### Copyright Notice:\n```\nading2210/vercel-llm-api: a reverse engineered API wrapper for the Vercel AI Playground\nCopyright (C) 2023 ading2210\n\nThis program is free software: you can redistribute it and/or modify\nit under the terms of the GNU General Public License as published by\nthe Free Software Foundation, either version 3 of the License, or\n(at your option) any later version.\n\nThis program is distributed in the hope that it will be useful,\nbut WITHOUT ANY WARRANTY; without even the implied warranty of\nMERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the\nGNU General Public License for more details.\n\nYou should have received a copy of the GNU General Public License\nalong with this program.  If not, see \u003chttps://www.gnu.org/licenses/\u003e.\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fading2210%2Fvercel-llm-api","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fading2210%2Fvercel-llm-api","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fading2210%2Fvercel-llm-api/lists"}