{"id":13577172,"url":"https://github.com/mobarski/aidapter","last_synced_at":"2025-10-06T10:32:29.766Z","repository":{"id":168820449,"uuid":"644618619","full_name":"mobarski/aidapter","owner":"mobarski","description":"Adapter / facade for language models (OpenAI, Anthropic, Cohere, local transformers, etc)","archived":false,"fork":false,"pushed_at":"2023-09-21T19:15:24.000Z","size":107,"stargazers_count":20,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-01-16T19:53:58.076Z","etag":null,"topics":["adapter","ai","anthropic","cohere","facade","llm","llms","openai","python","transformers"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mobarski.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2023-05-23T22:49:35.000Z","updated_at":"2024-12-07T21:06:24.000Z","dependencies_parsed_at":"2023-09-21T22:51:55.123Z","dependency_job_id":null,"html_url":"https://github.com/mobarski/aidapter","commit_stats":null,"previous_names":["mobarski/aidapter"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mobarski%2Faidapter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mobarski%2Faidapter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mobarski%2Faidapter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mobarski%2Faidapter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mobarski","download_url":"https://codeload.github.com/mobarski/aidapter/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":235519884,"owners_count":19003201,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["adapter","ai","anthropic","cohere","facade","llm","llms","openai","python","transformers"],"created_at":"2024-08-01T15:01:18.800Z","updated_at":"2025-10-06T10:32:29.433Z","avatar_url":"https://github.com/mobarski.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# aidapter\n\nSimple adapter for many language models -  remote (Hugging Face, OpenAI, AnthropicAI, CohereAI) and local (transformers library).\n\nFacilitates loading of many new models (Guanaco, Falcon, Vicuna, etc) in 16/8/4 bit modes.\n\nIt also supports embedding models (OpenAI, CohereAI, Sentence Transformers).\n\n## Installation\n\n:construction: This is experimental software. Anything can change without any notice.\n\n```\npip install git+https://github.com/mobarski/aidapter.git\n```\n\n**Note**: each vendor API requires manual installation of dependencies.\n\n## Features\n\n- simple, unified API to many models (remote and local)\n- batching\n- parallel calls\n- caching\n- usage tracking\n- automatic retries\n- response priming\n\n\n## Usage examples\n\n**completion:**\n\n```python\n\u003e\u003e\u003e import aidapter\n\u003e\u003e\u003e model = aidapter.model('openai:gpt-3.5-turbo') # uses OPENAI_API_KEY env variable\n\u003e\u003e\u003e model.complete('2+2=')\n4\n```\n\n```python\n\u003e\u003e\u003e model.complete(['2+2=','7*6=']) # parallel calls\n['4', '42']\n```\n\n**embeddings:**\n\n```python\n\u003e\u003e\u003e model = aidapter.model('sentence-transformers:multi-qa-mpnet-base-dot-v1')\n\u003e\u003e\u003e vector = model.embed('mighty indeed')\n\u003e\u003e\u003e vector[:5]\n[-0.07946087, -0.2150347, -0.33358946, 0.18340564, 0.16403404]\n```\n\n```python\n\u003e\u003e\u003e vectors = model.embed(['this is the way', 'so say we all']) # parallel / batch processing\n\u003e\u003e\u003e [x[:5] for x in vectors]\n[[0.037638217, -0.30608281, -0.3064257, -0.46715638, -0.2608084],\n [-0.063842215, -0.16669855, -0.22363697, -0.2893797, 0.060464755]]\n```\n\n**multiple models:**\n\n```python\n\u003e\u003e\u003e m1 = aidapter.model('transformers:ehartford/Wizard-Vicuna-13B-Uncensored:4bit') # 4 bit mode\n\u003e\u003e\u003e m2 = aidapter.model('anthropic:claude-instant-v1') # uses ANTHROPIC_API_KEY env variable\n```\n\n**persistent cache and usage tracking:**\n\n```python\n\u003e\u003e\u003e import shelve\n\u003e\u003e\u003e model.cache = shelve.open('/tmp/aidapter.cache') # persistant disk cache\n\u003e\u003e\u003e model.usage = shelve.open('/tmp/aidapter.usage') # persistant usage tracking\n```\n\n```python\n\u003e\u003e\u003e import diskcache as dc\n\u003e\u003e\u003e model.cache = dc.Cache('/tmp/aidapter.cache') # persistant disk cache\n\u003e\u003e\u003e model.usage = dc.Cache('/tmp/aidapter.usage') # persistant usage tracking\n```\n\n\n\n**function calling interface\\*:**\n\n```python\n\u003e\u003e\u003e def get_weather(city):\n\u003e\u003e\u003e     \"get weather info for a city; city must be all caps after ISO country code and a : separator (e.g. FR:PARIS)\"\n\u003e\u003e\u003e     ...\n\u003e\u003e\u003e model = aidapter.model('openai:gpt-3.5-turbo-0613')\n\u003e\u003e\u003e model.complete('Whats the weather in the capital of Poland?', functions=[get_weather])\n{'function_name': 'get_weather', 'arguments': {'city': 'PL:WARSAW'}}\n```\n\n\\* currently, it works only with selected OpenAI models\n\n**use last_hidden_state from any transformer as an embedding\\*:**\n\n```python\n\u003e\u003e\u003e model = aidapter.model('transformers:RWKV/rwkv-raven-1b5')\n\u003e\u003e\u003e model.raw_embed_one('mighty indeed')[:5]\n[0.14850381016731262, -0.021324729546904564, 0.09214707463979721, 0.34308338165283203, -0.11288302391767502]\n```\n\n\\* requires additional normalization over a corpus, API will change\n\n## API\n\n\n\naidapter.**model**(model_id, \\*\\*api_kwargs) **-\u003e model**\n\n- `model_id` - model identifier in the following format `\u003cvendor_name\u003e:\u003cmodel_name\u003e`\n- `api_kwargs` - default API arguments\n\n\n\nmodel.**complete**(prompt, system='', start='', stop=[], limit=100, temperature=0, functions=[], cache='use', debug=False) **-\u003e str | list | dict**\n\n- `prompt` - main prompt or list of prompts\n\n- `system` - system prompt\n\n- `start` - the text that will be appended to the start of the response and to the end of the prompt (aka response priming)\n\n- `stop` - list of strings upon which to stop generating\n\n- `limit` - maximum number of tokens to generate before stopping (aka max_new_tokens, max_tokens_to_sample)\n\n- `temperature` - amount of randomness\n\n- `functions` - list of functions available to the model (none of them will be executed - only the signatures are used)\n\n- `cache` - cache usage:\n  \n  - `use` - use the cache if the temperature is 0 (default)\n  - `skip` - don't use the cache\n  - `force` - use the cache even if the temperature is not 0\n  \n- `debug` - if True, the function will return a dictionary (or a list of dictionaries) containing internal objects / values\n\n  \n  \n  **FULL_PROMPT** = `system` + `prompt` + `start`\n  \n  \n\nmodel.**embed**(input, limit=None) -\u003e **list | list[list]**\n\n- `input` - text or list of texts\n- `limit` - limit the vector length to first n dimensions (default = None = no limit)\n\n\n\n**model configuration:**\n\n- `model.workers` - number of concurrent workers for parallel completion (default=4)\n\n- `model.show_progress` - show progress bar when performing parallel completion (default=False)\n\n- `model.retry_tries` - maximum number of retry attempts (default=5)\n\n- `model.retry_delay` - initial delay between retry attempts (default=0.1)\n\n- `model.retry_backoff` - multiplier applied to the delay between retry attempts (default=3)\n\n\n\n## Supported models\n\n### OpenAI\n\n- `openai:gpt-4`\n- `openai:gpt-4-32k`\n\n- `openai:gpt-3.5-turbo`\n\n- `openai:text-davinci-003`\n- `openai:code-davinci-002`\n- ...\n\nAPI key env. variable: **OPENAI_API_KEY**\n\n### Anthropic\n\n- `anthropic:claude-v1`\n\n- `anthropic:claude-instant-v1`\n\n- `anthropic:claude-v1-100k`\n\n- `anthropic:claude-instant-v1-100k`\n- ...\n\nAPI key env. variable: **ANTHROPIC_API_KEY**\n\n### Cohere\n\n- `cohere:command`\n\n- `cohere:command-light`\n- ...\n\nAPI key env. variable: **CO_API_KEY**\n\n### Transformers\n\n- `transformers:TheBloke/guanaco-7B-HF`\n\n- `transformers:tiiuae/falcon-7b`\n\n- `transformers:RWKV/rwkv-raven-3b`\n\n- `transformers:ehartford/Wizard-Vicuna-13B-Uncensored`\n\n- `transformers:roneneldan/TinyStories-33M`\n\n- ...\n\n  \n\n## Change log\n\n### 0.6.4\n\n- initial support for HF API models\n- removed old HF implementation\n\n### 0.6.3\n\n- OpenAI's embeddings now use BaseModelV2\n\n### 0.6.2\n\n- as_iter option in BaseModelV2.transform\n- removed BaseModelV2.register_progress\n\n### 0.6.1\n\n- handle cache=False in BaseModelV2.transform_many\n- hf2 brand renamed to huggingface\n\n### 0.6\n\n- initial support for HF API embeddings\n- BaseModelV2\n  - cleaner code\n  - diskcache support\n  - batch + threads support\n  - retry configuration\n  - progress update\n\n\n### 0.5.4\n\n- initial support for the functions argument (works only with selected OpenAI models)\n\n### 0.5.3\n\n- initial support for raw_embed_one in transformers (for creating embeddings from ANY transformer models)\n\n### 0.5.2\n\n- fix: kw handling in get_cache_key\n\n### 0.5.1\n\n- `limit` option for embedding models\n\n### 0.5\n\n- initial support for embedding models (requires more work with batch / parallel processing):\n  - OpenAI\n  - Cohere\n  - Sentence Transformers\n\n### 0.4.4\n\n- response priming (`start` option)\n\n### 0.4.3\n\n- `stop` option for transformers\n\n### 0.4.2\n\n- anthropic usage: tokens, characters\n\n- transformers usage: tokens, characters\n\n### 0.4.1\n\n- remove prompt from transformers output\n- removed kvdb\n- usage['time']\n- fixed pad_token_id\n- fixed limit in transformer models\n\n### 0.4\n\n- initial support for local transformers models\n\n  - float16 (add \":16bit\" to the model name)\n\n  - load_in_8bit (add \":8bit\" to the model name)\n\n  - load_in_4bit (add \":4bit\" to the model name)\n\n- cache = use | skip | force\n\n- shelve based persistence (for cache and usage)\n\n### 0.3.2\n\n- kvdb import fix\n\n### 0.3\n\n- Cohere models\n- disk cache\n\n### 0.2\n\n- OpenAI instruct models\n- Anthropic models (ANTHROPIC_API_KEY env variable)\n- complete: debug option\n- BaseModel.RENAME_KWARGS\n- pip install\n- limit handling\n\n### 0.1\n\n- parallel calls / cache / usage tracking / retries\n- OpenAI chat models\n\n\n\n### Next\n\n- HF API text generation\n- llama.cpp models (GGML!)\n- strangulate BaseModel with BaseModelV2\n\n\n\n## Reference Materials\n\n- https://github.com/kagisearch/pyllms\n- https://chat.lmsys.org/?leaderboard\n- https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard\n- https://huggingface.co/spaces/mteb/leaderboard\n- https://huggingface.co/blog/getting-started-with-embeddings\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmobarski%2Faidapter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmobarski%2Faidapter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmobarski%2Faidapter/lists"}