{"id":15757559,"url":"https://github.com/alterxyz/yvertexllm","last_synced_at":"2025-11-03T16:44:44.359Z","repository":{"id":251937477,"uuid":"838892168","full_name":"alterxyz/YVertexLLM","owner":"alterxyz","description":"Your Vertex AI - LLM cookbook","archived":false,"fork":false,"pushed_at":"2024-08-06T15:08:36.000Z","size":12,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-05-29T17:18:23.022Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/alterxyz.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-08-06T14:44:53.000Z","updated_at":"2024-08-06T15:08:40.000Z","dependencies_parsed_at":"2024-08-06T18:08:14.959Z","dependency_job_id":null,"html_url":"https://github.com/alterxyz/YVertexLLM","commit_stats":null,"previous_names":["alterxyz/yvertexllm"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/alterxyz/YVertexLLM","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alterxyz%2FYVertexLLM","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alterxyz%2FYVertexLLM/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alterxyz%2FYVertexLLM/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alterxyz%2FYVertexLLM/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/alterxyz","download_url":"https://codeload.github.com/alterxyz/YVertexLLM/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alterxyz%2FYVertexLLM/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265434842,"owners_count":23764594,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-04T09:22:10.537Z","updated_at":"2025-11-03T16:44:44.243Z","avatar_url":"https://github.com/alterxyz.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Your Vertex AI - LLM cookbook\n\nThis would be a good case for:\n\n- Try Vertex AI with Gemini\n- Using [Grounding](https://cloud.google.com/vertex-ai/generative-ai/docs/grounding/overview) at Generative AI on Vertex AI\n- Practice handling [Function calling](https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/function-calling) especially tools return\n    - Even with **stream** mode\n- Claude at Google Cloud Platform - Vertex AI\n- Real tool use / function calling with LongChain, DuckDuckGo Search, and Wolfram Alpha\n\n---\n\n## Vertex with Google Search\n\nA CLI based demo, allow you chat with Gemini with the [Grounding with Google Search](https://cloud.google.com/vertex-ai/generative-ai/docs/grounding/overview#ground-public)\n\n### Usage\n\n```bash\npython gemini_Vertex.py\n```\n\nThen open the generated html file in any browser to see results.\n\n### Setup\n\nSkip to the [Python environment](#python-environment) if you have already setup the gcloud CLI.\n\n#### gcloud CLI init\n\nIf you did not install and setup the gcloud, please make sure you download and install, by following the instruction in the [Google Cloud SDK](https://cloud.google.com/sdk/docs/install)\n\nFor initial setup after installing the gcloud, you need to run the following command to authenticate and set the project.\n\n```bash\ngcloud init\n# then you should login with your browser, choose the project, and set the region\n# then you many wanna restart the terminal / Windows system\ngcloud info # check if it's correct\n```\n\n#### Python environment\n\n```bash\npip install --upgrade google-cloud-aiplatform\ngcloud auth application-default login\n```\n\n### Note for Gemini\n\nEven with same project (in other llm provider, they may just made it simple as a single string of API key),\n\nthere is different from [Google AI Studio](https://aistudio.google.com/app/prompts/new_chat) and [Google Cloud Vertex AI](https://console.cloud.google.com/vertex-ai/generative/multimodal/create/text)\n\nThe difference you may see here are:\n\n### Google AI Studio\n\nSupport latest gemini mode `model_name=\"gemini-1.5-pro-exp-0801\"`\n\nSupport [Code execution](https://ai.google.dev/gemini-api/docs/code-execution?lang=python) which run python NumPy and SymPy,\nand it can not install your own libraries,\nand it run in google's sandbox so that codes may not safe won't run at your machine.\n\nIt's library is `pip install google-generativeai`,\n\nand *All You need is a API key* like other llm providers like openai or claude or cohere.\n\n### Google Cloud Vertex AI\n\nDoes not support the latest experimental mode, but has its own exp mode `gemini-experimental` with last update at 7/16/24.\n\nDoes not have code execution.\n\nSlighly different code style which require **gcloud** auth and project setup.\n\n---\n\n## Claude with Vertex AI with Tools Use (function calling)\n\n### Claude Usage\n\n```bash\npython gemini_Claude.py\n```\n\n### Perparation\n\nMake sure you have vaild Google Cloud Platform and granted the permission for Claude at Google Cloud:\n\n\u003chttps://console.cloud.google.com/vertex-ai/publishers/anthropic/model-garden/claude-3-5-sonnet\u003e\n\n### Setup_Claude\n\nIf you did not install and setup the gcloud, please go [here](#gcloud-cli-init) to install and setup the gcloud.\n\n```bash\npip install --upgrade google-cloud-aiplatform \"anthropic[vertex]\"\npip install -qU duckduckgo-search langchain-community\npip install wolframalpha\ngcloud auth application-default login\n```\n\n### Note for Claude and Tools Use\n\n### Tools\n\nFor Calculator tool, we picked the Wolfram Alpha as the tool, and for the Search tool, we picked the DuckDuckGo Search API. Those are both easy to get and use.\n\nThey are all good services provider, please do not abuse the services.\n\nFor Wolfram Alpha, you need to get the App ID from the [Wolfram Alpha](\u003chttps://developer.wolframalpha.com/\u003e)\n\nWhen getting an AppID, it is recommanded to choose the API type as \"Short Answers API\", since it will return *returns a single plain text result directly from [Wolfram|Alpha](https://products.wolframalpha.com/short-answers-api/documentation)*.\n\n### LongChain\n\nLongChain provides the Python SDK as a wrapper layer for those tools.\n\n### Multi-step Tool Use / Agents\n\nThe double call LLM that implement multi-step / auto / infinity tool use is inspired by Cohere's documentation.\n\nWhile implementing code locally, I use Wolfram Alpha as calculator instead, the preset structure of cohere's calculator was causing some issues.\n\nBut it also made great experience that:\n\nI saw the LLM did not get the correct answer as expected, and it try again at least 3 time with self improvement:\n\nFirst time it added the unit.\n\nSecond time it changed the format of the decimal places to natural language instead of quasi-mathematical language.\n\nThird time it adjusted again, because it returned the result but did not yet meet the user's requirements.\n\nI clearlly know it won't happen if there is Prompting with examples (One-, few-, and multi-shot), or Chain of Thought (CoT), or other prompt engineering, or make clear description at tool's defining part.\nBut if I did remember to do that, I probably won't realize and discover the self-improvement of the LLM.\n\n---\n\n## Ending\n\nI am so appreciated for those organizations and people who provide the great tools and services, and the great documentations and examples.\n\nSo I will do that too.\n\n## References\n\n[Ground responses for Gemini models](https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/ground-gemini#web-ground-gemini)\n\n[Tool use (function calling) - Anthropic](https://docs.anthropic.com/en/docs/build-with-claude/tool-use)\n\n[Vertex AI API - Anthropic](https://docs.anthropic.com/en/api/claude-on-vertex-ai)\n\n[LongChain DuckDuckGO Search](https://python.langchain.com/v0.2/docs/integrations/tools/ddg/)\n\n[LongChain Wolfram Alpha](https://python.langchain.com/v0.2/docs/integrations/tools/wolfram_alpha/)\n\n[DuckDuckGo Search API](https://duckduckgo.com/api)\n\n[Wolfram|Alpha Short Answers API](https://products.wolframalpha.com/short-answers-api/documentation)\n\n[Multi-step Tool Use (Agents)](https://docs.cohere.com/docs/multi-step-tool-use)\n\n## Additional Notes\n\nFeed back welcome, please open an issue or PR.\n\nThis document is almost **fully written by human**,\n\nwith a little help **but** just GitHub Copilot and markdown formatting at VSCode.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falterxyz%2Fyvertexllm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Falterxyz%2Fyvertexllm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falterxyz%2Fyvertexllm/lists"}