{"id":17187471,"url":"https://github.com/davidmchan/grazier","last_synced_at":"2025-04-13T19:08:33.883Z","repository":{"id":177023233,"uuid":"658952988","full_name":"DavidMChan/grazier","owner":"DavidMChan","description":"A tool for calling (and calling out to) large language models.","archived":false,"fork":false,"pushed_at":"2024-08-13T18:36:20.000Z","size":110,"stargazers_count":16,"open_issues_count":0,"forks_count":2,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-08T01:47:41.003Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DavidMChan.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-06-26T21:02:26.000Z","updated_at":"2025-03-29T15:53:21.000Z","dependencies_parsed_at":"2023-12-19T00:45:51.246Z","dependency_job_id":"7778e150-e42b-427c-aacf-39e38e4cdd3f","html_url":"https://github.com/DavidMChan/grazier","commit_stats":null,"previous_names":["davidmchan/grazier"],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DavidMChan%2Fgrazier","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DavidMChan%2Fgrazier/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DavidMChan%2Fgrazier/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DavidMChan%2Fgrazier/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DavidMChan","download_url":"https://codeload.github.com/DavidMChan/grazier/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248766735,"owners_count":21158301,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-15T01:06:28.789Z","updated_at":"2025-04-13T19:08:33.850Z","avatar_url":"https://github.com/DavidMChan.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Grazier: Easily call Large Language Models from a unified API\n\nGrazier is a Python library for easily calling large language models from a unified API.\n\n## Supported Large Models\n\nFrom OpenAI:\n- GPT-4 (Base, 32K) (Chat and Completion Engines)\n- GPT-3.5 (ChatGPT) (Chat and Completion Engines)\n- GPT-3 (Davinci (v2,v3), Ada, Babbage, Curie) (Completion Engine)\n\nFrom Anthropic:\n- Claude 2 (Base) (Chat and Completion Engines)\n- Claude (Base, 100K) (Chat and Completion Engines)\n- Claude Instant (Base, 100K) (Chat and Completion Engines)\n\nFrom Google/GCP:\n- PaLM (Chat and Completion Engines)\n\nFrom Huggingface\n- GPT-2 (Base, Medium, Large, XL) (Completion Engine)\n- GPT-Neo (125M, 1.3B, 2.7B) (Completion Engine)\n- GPT-J (6B) (Completion Engine)\n- Falcon (7B, 40B, rw-1B, rw-7B) (Completion Engine)\n- Dolly (v1 - 6B, v2 - 3B, 7B, 12B) (Chat and Completion Engines)\n- MPT (Instruct - 7B, 30B) (Chat and Completion Engines)\n\nFrom Facebook (via Huggingface)\n- Llama (7B, 13B, 30B, 65B) (Completion Engine)\n- Llama 2 (7B, 13B, 70B) (Completion Engine)\n- OPT (125M, 350M, 1.3B, 2.7B, 6.7B, 13B, 30B, 66B) (Completion Engine)\n\nFrom Stanford (via Huggingface)\n- Alpaca (7B) (Chat and Completion Engines)\n\nFrom Berkeley (via Huggingface)\n- Koala (7B, 13B_v1, 13B_v2) (Chat and Completion Engines)\n- Vicuna (7B, 13B) (Chat and Completion Engines)\n\nFrom StabilityAI (via Huggingface)\n- StableLM (7B, 13B) (Chat and Completion Engines)\n\nFrom AllenAI (via Huggingface)\n- Tulu (7B, 13B, 30B, 65B) (Chat and Completion Engines)\n- Open Instruct (ShareGPT) (7B, 13B, 30B, 65B) (Chat and Completion Engines)\n\nFrom AI21\n- Jurassic 2 (Light, Mid, Ultra) (Completion Engines)\n\nImage/Language Models:\n- Blip2 (2.7B, 6.7B) (Image + Text Engine)\n- Open Flamingo (3B, 4B, 9B) (Image + Text Engine)\n\n\n## Installation\n\nGrazier can easily be installed using pip:\n```bash\npip install grazier\n```\n\nEach of the LLMs may need additional setup, which you can find in the engine setup section below.\n\n\n## Usage\n\nFor completion engines, it's as simple as:\n```python\nimport grazier\n\ngrazier.list_models()\n['gptj-6B', 'gpt2', 'gpt2-med', 'gpt2-lg', 'gpt2-xl', 'distilgpt2', 'gptneo-125M', 'gptneo-1.3B', 'gptneo-2.7B', 'stablelm-3B', 'stablelm-7B', 'opt-125M', 'opt-350M', 'opt-1.3b', 'opt-2.7b', 'opt-6.7b', 'opt-13b', 'opt-30b', 'opt-66b', 'llama-7B', 'llama-13B', 'llama-30B', 'llama-65B', 'gpt3-davinci3', 'gpt3-davinci2', 'gpt3-curie', 'gpt3-babbage', 'gpt3-ada', 'palm']\ngpt2 = grazier.get(\"gpt2\")\ncompletion = gpt2(\"I enjoy walking with my cute dog, but sometimes he gets scared and\")\nprint(completion)\n```\n\nFor chat engines, all you need to do is add the `type=\"chat\"` parameter:\n```python\nfrom grazier import Conversation, Speaker, get, list_models\n\nconversation = Conversation()\nconversation.add_turn(\"You are a funny person.\", speaker=Speaker.SYSTEM)\nconversation.add_turn(\"Hi, how are you?\", speaker=Speaker.USER)\nconversation.add_turn(\"I am doing well, how about you?\", speaker=Speaker.AI)\nconversation.add_turn(\"What are you planning to do today?\", speaker=Speaker.USER)\n\nlist_models(type=\"chat\")\n['claude', 'claude-100k', 'claude-instant', 'claude-instant-100k', 'bard', 'koala-7b', 'koala-13b-v1', 'koala-13b-v2', 'vicuna-7b', 'vicuna-13b', 'alpaca-13b', 'chat-gpt', 'gpt4', 'gpt4-32k', 'stablelm-3b', 'stablelm-7b', 'palm']\ngpt4 = get(\"gpt4\", type=\"chat\")\nnext_turn = gpt4(conversation)\nprint(next_turn)\n```\n\nFor vision-augmented (image) engines, use `type=\"image\"`\n```python\nimport grazier\nfrom PIL import Image\n\ngrazier.list_models(type=\"image\")\n['blip2-opt-2.7b', 'blip2-opt-6.7b', 'blip2-opt-2.7b-coco', 'blip2-opt-6.7b-coco', 'blip2-flan-t5-xl', 'blip2-flan-t5-xxl', 'blip2-flan-t5-xl-coco', 'openflamingo-3b-vitl-mpt1b', 'openflamingo-3b-vitl-mpt1b-dolly', 'openflamingo-9b-vitl-mpt7b', 'openflamingo-4b-vitl-rpj3b']\nblip2 = grazier.get(\"blip2-opt-2.7b\", type=\"image\")\n\nimage = Image.open('test_data/dog.jpg')\ncompletion = blip2(image, \"A photo of\")\n\nprint(completion)\n```\n\n## Individual Engine Setup\n\nEach engine may require some specific details to be passed in. For example, OpenAI engines require an API key. These\ndetails are generally set up with environment variables.\n\n### OpenAI Engines\n\nFor OpenAI engines, you will need to set the `OPENAI_API_KEY` and `OPENAI_API_ORG` environment variables. You can find\nyour API key and organization ID on the [OpenAI dashboard](https://platform.openai.com/). You can set these environment\nvariables in your shell or in a `.env` file in the root of your project. For example, in a `.env` file, you would have:\n```bash\nOPENAI_API_KEY=\u003cyour key\u003e\nOPENAI_API_ORG=\u003cyour org id\u003e\n```\nor on the command line:\n```bash\nexport OPENAI_API_KEY=\u003cyour key\u003e\nexport OPENAI_API_ORG=\u003cyour org id\u003e\n```\n\n### Anthropic Engines\nFor Anthropic engines, you will need to set the `ANTHROPIC_API_KEY` environment variable. You can find your API key at\nthe [Anthropic dashboard](https://console.anthropic.com/account/keys). You can set this environment variable in\nyour shell or in a `.env` file in the root of your project. For example, in a `.env` file, you would have:\n```bash\nANTHROPIC_API_KEY=\u003cyour key\u003e\n```\nor on the command line:\n```bash\nexport ANTHROPIC_API_KEY=\u003cyour key\u003e\n```\n\n### Vertex Engines (PaLM)\nFor Google engines, we use the Vertex cloud API, which requires a Google Cloud Platform (GCP) project. You can create a\nGCP project at the [GCP console](https://console.cloud.google.com/). You will also need to enable the Vertex AI API for\nyour project, set up a services account, and download the account JSON credentials. You can find instructions for that\nfollowing steps 1 to 6 of the tutorial  [here](https://cloud.google.com/vertex-ai/docs/tutorials/image-recognition-automl).\nFinally, you will need to set the `GOOGLE_APPLICATION_CREDENTIALS` environment variable to the path of the JSON file.\nYou can set this environment variable in your shell or in a `.env` file in the root of your project. For example, in a\n`.env` file, you would have:\n```bash\nGOOGLE_APPLICATION_CREDENTIALS=\u003cpath to your JSON file\u003e\n```\nor on the command line:\n```bash\nexport GOOGLE_APPLICATION_CREDENTIALS=\u003cpath to your JSON file\u003e\n```\n\n### Bard\nFor the Bard engine, you will need to get your Bard __Secure-1PSID and __Secure-1PSIDTS tokens.  Get the value of this\nvariable by first going to https://bard.google.com/, then log in, press F12 for console, and go to the \"Application\" tab,\nthen \"Cookies\", then copy the value of the \"__Secure-1PSID\" and \"__Secure-1PSIDTS\" cookies. You can then set the\nenvironment variables:\n```bash\nBARD__Secure_1PSID=\u003cyour session id\u003e\nBARD__Secure_1PSIDTS=\u003cyour session id timestamp\u003e\n```\n\n### Huggingface Engines\nMost of the huggingface engines require no additional setup, however, some of the larger models require a GPU to run\nwith any kind of efficiency (and some require multiple GPUs with large amounts of memory). You can find more details\nabout the requirements for each model on the [Huggingface model hub](https://huggingface.co/models).\n\n### Llama, Alpaca, Koala, Vicuna and AllenAI Engines\nFor these engines, you will need to obtain and postprocess the weights yourself (due to Facebook's licensing). You can\nfind the instructions for doing so on each model page:\n- Llama: https://huggingface.co/docs/transformers/main/model_doc/llama\n- Alpaca: https://github.com/tatsu-lab/stanford_alpaca#recovering-alpaca-weights\n- Koala: https://github.com/young-geng/EasyLM/blob/main/docs/koala.md\n- Vicuna: https://github.com/lm-sys/FastChat#vicuna-weights\n- AllenAI: https://huggingface.co/allenai/tulu-65b\n\nOnce the weights have been downloaded and processed, you can set the following environment variables to the root\ndirectory containing a folder for each variant (The format is, `{root_dir}/{model-prefix}/weights.bin`, the root directory would\nbe `root_dir`, and the model-prefix is the name of the model, e.g. `tulu-65b`):\n```bash\nLLAMA_WEIGHTS_ROOT=\u003cpath to the llama weights\u003e\nALPACA_WEIGHTS_ROOT=\u003cpath to the alpaca weights\u003e\nKOALA_WEIGHTS_ROOT=\u003cpath to the koala weights\u003e\nVICUNA_WEIGHTS_ROOT=\u003cpath to the vicuna weights\u003e\nALLENAI_WEIGHTS_ROOT=\u003cpath to the allenai weights\u003e\n```\n\n### AI21 Models (Jurassic)\nFor AI21 models, you will need to set the `AI21_API_KEY` environment variable. You can find your API key at\nthe [AI21 Studio Dashboard](https://studio.ai21.com/account/api-key). You can set this environment variable in\nyour shell or in a `.env` file in the root of your project. For example, in a `.env` file, you would have:\n```bash\nAI21_API_KEY=\u003cyour key\u003e\n```\nor on the command line:\n```bash\nexport AI21_API_KEY=\u003cyour key\u003e\n```\n\n## Citation\n\nIf you use grazier in your work, please cite:\n\n```\n@misc{grazier,\n  author = {David Chan},\n  title = {grazier: Easily call Large Language Models from a unified API},\n  year = {2023},\n  publisher = {GitHub},\n  journal = {GitHub repository},\n  howpublished = {\\url{\n    https://github.com/DavidMChan/grazier\n  }}\n}\n```\n\n## License\n\ngrazier is licensed under the terms of the MIT license. See [LICENSE](LICENSE) for more information.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdavidmchan%2Fgrazier","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdavidmchan%2Fgrazier","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdavidmchan%2Fgrazier/lists"}