{"id":13564158,"url":"https://github.com/r2d4/openlm","last_synced_at":"2025-04-04T14:06:58.512Z","repository":{"id":163594142,"uuid":"637986780","full_name":"r2d4/openlm","owner":"r2d4","description":"OpenAI-compatible Python client that can call any LLM","archived":false,"fork":false,"pushed_at":"2023-05-19T16:41:55.000Z","size":20,"stargazers_count":366,"open_issues_count":1,"forks_count":22,"subscribers_count":6,"default_branch":"main","last_synced_at":"2024-10-30T04:49:19.731Z","etag":null,"topics":["cohere","huggingface","llm","openai"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/r2d4.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2023-05-08T20:54:57.000Z","updated_at":"2024-10-06T13:04:25.000Z","dependencies_parsed_at":"2024-01-06T21:43:25.543Z","dependency_job_id":"390bb1be-8eb2-4953-8a29-f35826c9100a","html_url":"https://github.com/r2d4/openlm","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/r2d4%2Fopenlm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/r2d4%2Fopenlm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/r2d4%2Fopenlm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/r2d4%2Fopenlm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/r2d4","download_url":"https://codeload.github.com/r2d4/openlm/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247190250,"owners_count":20898702,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cohere","huggingface","llm","openai"],"created_at":"2024-08-01T13:01:27.343Z","updated_at":"2025-04-04T14:06:58.486Z","avatar_url":"https://github.com/r2d4.png","language":"Python","funding_links":[],"categories":["Python","其他LLM框架","Other LLM Frameworks"],"sub_categories":["文章","Videos Playlists"],"readme":"# OpenLM\n\nDrop-in OpenAI-compatible library that can call LLMs from other providers (e.g., HuggingFace, Cohere, and more). \n\n```diff\n1c1\n\u003c import openai\n---\n\u003e import openlm as openai\n\ncompletion = openai.Completion.create(\n    model=[\"bloom-560m\", \"cohere.ai/command\"], \n    prompt=[\"Hello world!\", \"A second prompt!\"]\n)\nprint(completion)\n```\n### Features\n* Takes in the same parameters as OpenAI's Completion API and returns a similarly structured response. \n* Call models from HuggingFace's inference endpoint API, Cohere.ai, OpenAI, or your custom implementation. \n* Complete multiple prompts on multiple models in the same request. \n* Very small footprint: OpenLM calls the inference APIs directly rather than using multiple SDKs.\n\n\n### Installation\n```bash\npip install openlm\n```\n\n### Examples\n\n- [Import as OpenAI](examples/as_openai.py)\n- [Set up API keys via environment variables or pass a dict](examples/api_keys.py)\n- [Add a custom model or provider](examples/custom_provider.py)\n- [Complete multiple prompts on multiple models](examples/multiplex.py)\n\nOpenLM currently supports the Completion endpoint, but over time will support more standardized endpoints that make sense. \n\n### [Example with Response](examples/multiplex.py)\n\n```python\nimport sys\nfrom pathlib import Path\n\nsys.path.append(str(Path(__file__).resolve().parent.parent))\n\nimport openlm \nimport json\n\ncompletion = openlm.Completion.create(\n    model=[\"ada\", \"huggingface.co/gpt2\", \"cohere.ai/command\"],\n    prompt=[\"The quick brown fox\", \"Who jumped over the lazy dog?\"],\n    max_tokens=15\n)\nprint(json.dumps(completion, indent=4))\n```\n\n```json\n{\n    \"id\": \"504cc502-dc27-43e7-bcc3-b62e178c247e\",\n    \"object\": \"text_completion\",\n    \"created\": 1683583267,\n    \"choices\": [\n        {\n            \"id\": \"c0487ba2-935d-4dec-b191-f7eff962f117\",\n            \"model_idx\": 0,\n            \"model_name\": \"openai.com/ada\",\n            \"index\": 0,\n            \"created\": 1683583233,\n            \"text\": \" jumps into the much bigger brown bush.\\\" \\\"Alright, people like you can\",\n            \"usage\": {\n                \"prompt_tokens\": 4,\n                \"completion_tokens\": 15,\n                \"total_tokens\": 19\n            },\n            \"extra\": {\n                \"id\": \"cmpl-7E3CCSpJHXfx5yB0TaJU9ON7rNYPT\"\n            }\n        },\n        {\n            \"id\": \"bab92d11-5ba6-4da2-acca-1f3398a78c3e\",\n            \"model_idx\": 0,\n            \"model_name\": \"openai.com/ada\",\n            \"index\": 1,\n            \"created\": 1683583233,\n            \"text\": \"\\n\\nIt turns out that saying one's name \\\"Joe\\\" is the\",\n            \"usage\": {\n                \"prompt_tokens\": 7,\n                \"completion_tokens\": 15,\n                \"total_tokens\": 22\n            },\n            \"extra\": {\n                \"id\": \"cmpl-7E3CDBbqFy92I2ZbSGoDT5ickAiPD\"\n            }\n        },\n        {\n            \"id\": \"be870636-9d9e-4f74-b8bd-d04766072a7b\",\n            \"model_idx\": 1,\n            \"model_name\": \"huggingface.co/gpt2\",\n            \"index\": 0,\n            \"created\": 1683583234,\n            \"text\": \"The quick brown foxes, and the short, snuggly fox-scented, soft foxes we have in our household\\u2026 all come in two distinct flavours: yellow and orange; and red and white. This mixture is often confused with\"\n        },\n        {\n            \"id\": \"c1abf535-54a9-4b72-8681-d3b4a601da88\",\n            \"model_idx\": 1,\n            \"model_name\": \"huggingface.co/gpt2\",\n            \"index\": 1,\n            \"created\": 1683583266,\n            \"text\": \"Who jumped over the lazy dog? He probably got it, but there's only so much you do when you lose one.\\n\\nBut I will say for a moment that there's no way this guy might have picked a fight with Donald Trump.\"\n        },\n        {\n            \"id\": \"08e8c351-236a-4497-98f3-488cdc0b6b6a\",\n            \"model_idx\": 2,\n            \"model_name\": \"cohere.ai/command\",\n            \"index\": 0,\n            \"created\": 1683583267,\n            \"text\": \"\\njumps over the lazy dog.\",\n            \"extra\": {\n                \"request_id\": \"0bbb28c0-eb3d-4614-b4d9-1eca88c361ca\",\n                \"generation_id\": \"5288dd6f-3ecf-475b-b909-0b226be6a193\"\n            }\n        },\n        {\n            \"id\": \"49ce51e6-9a18-4093-957f-54a1557c8829\",\n            \"model_idx\": 2,\n            \"model_name\": \"cohere.ai/command\",\n            \"index\": 1,\n            \"created\": 1683583267,\n            \"text\": \"\\nThe quick brown fox.\",\n            \"extra\": {\n                \"request_id\": \"ab5d5e03-22a1-42cd-85b2-9b9704c79304\",\n                \"generation_id\": \"60493966-abf6-483c-9c47-2ea5c5eeb855\"\n            }\n        }\n    ],\n    \"usage\": {\n        \"prompt_tokens\": 11,\n        \"completion_tokens\": 30,\n        \"total_tokens\": 41\n    }\n}\n```\n\n### Other Languages\n[r2d4/llm.ts](https://github.com/r2d4/llm.ts) is a TypeScript library that has a similar API that sits on top of multiple language models.\n\n### Roadmap\n- [ ] Streaming API\n- [ ] Embeddings API\n\n### Contributing\nContributions are welcome! Please open an issue or submit a PR.\n\n### License\n[MIT](LICENSE)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fr2d4%2Fopenlm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fr2d4%2Fopenlm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fr2d4%2Fopenlm/lists"}