{"id":13628239,"url":"https://github.com/gbaptista/gemini-ai","last_synced_at":"2025-04-05T20:07:47.530Z","repository":{"id":212565587,"uuid":"731803438","full_name":"gbaptista/gemini-ai","owner":"gbaptista","description":"A Ruby Gem for interacting with Gemini through Vertex AI, Generative Language API, or AI Studio, Google's generative AI services.","archived":false,"fork":false,"pushed_at":"2024-07-21T10:48:41.000Z","size":186,"stargazers_count":97,"open_issues_count":9,"forks_count":21,"subscribers_count":5,"default_branch":"main","last_synced_at":"2024-10-11T18:57:53.047Z","etag":null,"topics":["ai","deepmind","gemini","gemini-ai","gemini-functions","gemini-pro","gemini-tools","google","google-ai","google-gemini","google-vertex-ai","llm","nano-bots","vertex","vertex-ai"],"latest_commit_sha":null,"homepage":"https://rubygems.org/gems/gemini-ai","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/gbaptista.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-12-14T23:14:06.000Z","updated_at":"2024-10-04T20:09:27.000Z","dependencies_parsed_at":"2024-01-13T14:21:57.088Z","dependency_job_id":"77daaa8b-d741-47a5-9448-e9d28dcbfb66","html_url":"https://github.com/gbaptista/gemini-ai","commit_stats":null,"previous_names":["gbaptista/gemini-ai"],"tags_count":13,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gbaptista%2Fgemini-ai","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gbaptista%2Fgemini-ai/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gbaptista%2Fgemini-ai/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gbaptista%2Fgemini-ai/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/gbaptista","download_url":"https://codeload.github.com/gbaptista/gemini-ai/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247393570,"owners_count":20931812,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","deepmind","gemini","gemini-ai","gemini-functions","gemini-pro","gemini-tools","google","google-ai","google-gemini","google-vertex-ai","llm","nano-bots","vertex","vertex-ai"],"created_at":"2024-08-01T22:00:48.983Z","updated_at":"2025-04-05T20:07:47.510Z","avatar_url":"https://github.com/gbaptista.png","language":"Ruby","funding_links":[],"categories":["GitHub projects","Ruby"],"sub_categories":[],"readme":"# Gemini AI\n\nA Ruby Gem for interacting with [Gemini](https://deepmind.google/technologies/gemini/) through [Vertex AI](https://cloud.google.com/vertex-ai), [Generative Language API](https://ai.google.dev/api/rest), or [AI Studio](https://makersuite.google.com), Google's generative AI services.\n\n![The logo shows a gemstone split into red and blue halves, symbolizing Ruby programming and Gemini AI. It's surrounded by a circuit-like design on a dark blue backdrop.](https://raw.githubusercontent.com/gbaptista/assets/main/gemini-ai/ruby-gemini-ai.png)\n\n\u003e _This Gem is designed to provide low-level access to Gemini, enabling people to build abstractions on top of it. If you are interested in more high-level abstractions or more user-friendly tools, you may want to consider [Nano Bots](https://github.com/icebaker/ruby-nano-bots) 💎 🤖._\n\n## TL;DR and Quick Start\n\n```ruby\ngem 'gemini-ai', '~\u003e 4.2.0'\n```\n\n```ruby\nrequire 'gemini-ai'\n\n# With an API key\nclient = Gemini.new(\n  credentials: {\n    service: 'generative-language-api',\n    api_key: ENV['GOOGLE_API_KEY']\n  },\n  options: { model: 'gemini-pro', server_sent_events: true }\n)\n\n# With a Service Account Credentials File\nclient = Gemini.new(\n  credentials: {\n    service: 'vertex-ai-api',\n    file_path: 'google-credentials.json',\n    region: 'us-east4'\n  },\n  options: { model: 'gemini-pro', server_sent_events: true }\n)\n\n# With the Service Account Credentials File contents\nclient = Gemini.new(\n  credentials: {\n    service: 'vertex-ai-api',\n    file_contents: File.read('google-credentials.json'),\n    # file_contents: ENV['GOOGLE_CREDENTIALS_FILE_CONTENTS'],\n    region: 'us-east4'\n  },\n  options: { model: 'gemini-pro', server_sent_events: true }\n)\n\n# With Application Default Credentials\nclient = Gemini.new(\n  credentials: {\n    service: 'vertex-ai-api',\n    region: 'us-east4'\n  },\n  options: { model: 'gemini-pro', server_sent_events: true }\n)\n\nresult = client.stream_generate_content({\n  contents: { role: 'user', parts: { text: 'hi!' } }\n})\n```\n\nResult:\n```ruby\n[{ 'candidates' =\u003e\n   [{ 'content' =\u003e {\n        'role' =\u003e 'model',\n        'parts' =\u003e [{ 'text' =\u003e 'Hello! How may I assist you?' }]\n      },\n      'finishReason' =\u003e 'STOP',\n      'safetyRatings' =\u003e\n      [{ 'category' =\u003e 'HARM_CATEGORY_HARASSMENT', 'probability' =\u003e 'NEGLIGIBLE' },\n       { 'category' =\u003e 'HARM_CATEGORY_HATE_SPEECH', 'probability' =\u003e 'NEGLIGIBLE' },\n       { 'category' =\u003e 'HARM_CATEGORY_SEXUALLY_EXPLICIT', 'probability' =\u003e 'NEGLIGIBLE' },\n       { 'category' =\u003e 'HARM_CATEGORY_DANGEROUS_CONTENT', 'probability' =\u003e 'NEGLIGIBLE' }] }],\n   'usageMetadata' =\u003e {\n     'promptTokenCount' =\u003e 2,\n     'candidatesTokenCount' =\u003e 8,\n     'totalTokenCount' =\u003e 10\n   } }]\n```\n\n## Index\n\n- [TL;DR and Quick Start](#tldr-and-quick-start)\n- [Index](#index)\n- [Setup](#setup)\n  - [Installing](#installing)\n  - [Credentials](#credentials)\n    - [Option 1: API Key (Generative Language API)](#option-1-api-key-generative-language-api)\n    - [Option 2: Service Account Credentials File (Vertex AI API)](#option-2-service-account-credentials-file-vertex-ai-api)\n    - [Option 3: Application Default Credentials (Vertex AI API)](#option-3-application-default-credentials-vertex-ai-api)\n    - [Required Data](#required-data)\n  - [Custom Version](#custom-version)\n- [Available Models](#available-models)\n- [Usage](#usage)\n  - [Client](#client)\n  - [Methods](#methods)\n    - [Chat](#chat)\n      - [stream_generate_content](#stream_generate_content)\n        - [Receiving Stream Events](#receiving-stream-events)\n        - [Without Events](#without-events)\n      - [generate_content](#generate_content)\n    - [Embeddings](#embeddings)\n      - [predict](#predict)\n      - [embed_content](#embed_content)\n  - [Modes](#modes)\n    - [Text](#text)\n    - [Image](#image)\n    - [Video](#video)\n  - [Streaming vs. Server-Sent Events (SSE)](#streaming-vs-server-sent-events-sse)\n    - [Server-Sent Events (SSE) Hang](#server-sent-events-sse-hang)\n    - [Non-Streaming](#non-streaming)\n  - [Back-and-Forth Conversations](#back-and-forth-conversations)\n  - [Safety Settings](#safety-settings)\n  - [System Instructions](#system-instructions)\n  - [Counting Tokens](#counting-tokens)\n  - [JSON Format Responses](#json-format-responses)\n    - [JSON Schema](#json-schema)\n    - [Models That Support JSON](#models-that-support-json)\n  - [Tools (Functions) Calling](#tools-functions-calling)\n  - [New Functionalities and APIs](#new-functionalities-and-apis)\n  - [Request Options](#request-options)\n    - [Adapter](#adapter)\n    - [Timeout](#timeout)\n  - [Error Handling](#error-handling)\n    - [Rescuing](#rescuing)\n    - [For Short](#for-short)\n    - [Errors](#errors)\n- [Development](#development)\n  - [Purpose](#purpose)\n  - [Publish to RubyGems](#publish-to-rubygems)\n  - [Updating the README](#updating-the-readme)\n- [Resources and References](#resources-and-references)\n- [Disclaimer](#disclaimer)\n\n## Setup\n\n### Installing\n\n```sh\ngem install gemini-ai -v 4.2.0\n```\n\n```sh\ngem 'gemini-ai', '~\u003e 4.2.0'\n```\n\n### Credentials\n\n- [Option 1: API Key (Generative Language API)](#option-1-api-key-generative-language-api)\n- [Option 2: Service Account Credentials File (Vertex AI API)](#option-2-service-account-credentials-file-vertex-ai-api)\n- [Option 3: Application Default Credentials (Vertex AI API)](#option-3-application-default-credentials-vertex-ai-api)\n- [Required Data](#required-data)\n\n\u003e ⚠️ DISCLAIMER: Be careful with what you are doing, and never trust others' code related to this. These commands and instructions alter the level of access to your Google Cloud Account, and running them naively can lead to security risks as well as financial risks. People with access to your account can use it to steal data or incur charges. Run these commands at your own responsibility and due diligence; expect no warranties from the contributors of this project.\n\n#### Option 1: API Key (Generative Language API)\n\nYou need a [Google Cloud](https://console.cloud.google.com) [_Project_](https://cloud.google.com/resource-manager/docs/creating-managing-projects), and then you can generate an API Key through the Google Cloud Console [here](https://console.cloud.google.com/apis/credentials).\n\nYou also need to enable the _Generative Language API_ service in your Google Cloud Console, which can be done [here](https://console.cloud.google.com/apis/library/generativelanguage.googleapis.com).\n\n\nAlternatively, you can generate an API Key through _Google AI Studio_ [here](https://makersuite.google.com/app/apikey). However, this approach will automatically create a project for you in your Google Cloud Account.\n\n#### Option 2: Service Account Credentials File (Vertex AI API)\n\nYou need a [Google Cloud](https://console.cloud.google.com) [_Project_](https://cloud.google.com/resource-manager/docs/creating-managing-projects) and a [_Service Account_](https://cloud.google.com/iam/docs/service-account-overview) to use [Vertex AI](https://cloud.google.com/vertex-ai) API.\n\nAfter creating them, you need to enable the Vertex AI API for your project by clicking `Enable` here: [Vertex AI API](https://console.cloud.google.com/apis/library/aiplatform.googleapis.com).\n\nYou can create credentials for your _Service Account_ [here](https://console.cloud.google.com/apis/credentials), where you will be able to download a JSON file named `google-credentials.json` that should have content similar to this:\n\n```json\n{\n  \"type\": \"service_account\",\n  \"project_id\": \"YOUR_PROJECT_ID\",\n  \"private_key_id\": \"a00...\",\n  \"private_key\": \"-----BEGIN PRIVATE KEY-----\\n...\\n-----END PRIVATE KEY-----\\n\",\n  \"client_email\": \"PROJECT_ID@PROJECT_ID.iam.gserviceaccount.com\",\n  \"client_id\": \"000...\",\n  \"auth_uri\": \"https://accounts.google.com/o/oauth2/auth\",\n  \"token_uri\": \"https://oauth2.googleapis.com/token\",\n  \"auth_provider_x509_cert_url\": \"https://www.googleapis.com/oauth2/v1/certs\",\n  \"client_x509_cert_url\": \"https://www.googleapis.com/robot/v1/metadata/x509/...\"\n}\n```\n\nYou need to have the necessary [policies](https://cloud.google.com/iam/docs/policies) (`roles/aiplatform.user` and possibly `roles/ml.admin`) in place to use the Vertex AI API.\n\nYou can add them by navigating to the [IAM Console](https://console.cloud.google.com/iam-admin/iam) and clicking on the _\"Edit principal\"_ (✏️ pencil icon) next to your _Service Account_.\n\nAlternatively, you can add them through the [gcloud CLI](https://cloud.google.com/sdk/gcloud) as follows:\n\n```sh\ngcloud projects add-iam-policy-binding PROJECT_ID \\\n  --member='serviceAccount:PROJECT_ID@PROJECT_ID.iam.gserviceaccount.com' \\\n  --role='roles/aiplatform.user'\n```\n\nSome people reported having trouble accessing the API, and adding the role `roles/ml.admin` fixed it:\n\n```sh\ngcloud projects add-iam-policy-binding PROJECT_ID \\\n  --member='serviceAccount:PROJECT_ID@PROJECT_ID.iam.gserviceaccount.com' \\\n  --role='roles/ml.admin'\n```\n\nIf you are not using a _Service Account_:\n```sh\ngcloud projects add-iam-policy-binding PROJECT_ID \\\n  --member='user:YOUR@MAIL.COM' \\\n  --role='roles/aiplatform.user'\n\ngcloud projects add-iam-policy-binding PROJECT_ID \\\n  --member='user:YOUR@MAIL.COM' \\\n  --role='roles/ml.admin'\n```\n\n#### Option 3: Application Default Credentials (Vertex AI API)\n\nSimilar to [Option 2](#option-2-service-account-credentials-file-vertex-ai-api), but you don't need to download a `google-credentials.json`. [_Application Default Credentials_](https://cloud.google.com/docs/authentication/application-default-credentials) automatically find credentials based on the application environment.\n\nFor local development, you can generate your default credentials using the [gcloud CLI](https://cloud.google.com/sdk/gcloud) as follows:\n\n```sh\ngcloud auth application-default login\n```\n\nFor more details about alternative methods and different environments, check the official documentation:\n[Set up Application Default Credentials](https://cloud.google.com/docs/authentication/provide-credentials-adc)\n\n#### Required Data\n\nAfter choosing an option, you should have all the necessary data and access to use Gemini.\n\n**Option 1**, for API Key:\n\n```ruby\n{\n  service: 'generative-language-api',\n  api_key: 'GOOGLE_API_KEY'\n}\n```\n\nRemember that hardcoding your API key in code is unsafe; it's preferable to use environment variables:\n\n```ruby\n{\n  service: 'generative-language-api',\n  api_key: ENV['GOOGLE_API_KEY']\n}\n```\n\n**Option 2**: For the Service Account, provide a `google-credentials.json` file and a `REGION`:\n\n```ruby\n{\n  service: 'vertex-ai-api',\n  file_path: 'google-credentials.json',\n  region: 'us-east4'\n}\n```\n\nAlternatively, you can pass the file contents instead of the path:\n```ruby\n{\n  service: 'vertex-ai-api',\n  file_contents: File.read('google-credentials.json'),\n  region: 'us-east4'\n}\n```\n\n```ruby\n{\n  service: 'vertex-ai-api',\n  file_contents: ENV['GOOGLE_CREDENTIALS_FILE_CONTENTS'],\n  region: 'us-east4'\n}\n```\n\n**Option 3**: For _Application Default Credentials_, omit both the `api_key` and the `file_path`:\n\n```ruby\n{\n  service: 'vertex-ai-api',\n  region: 'us-east4'\n}\n```\n\nAs of the writing of this README, the following regions support Gemini:\n```text\nIowa (us-central1)\nLas Vegas, Nevada (us-west4)\nMontréal, Canada (northamerica-northeast1)\nNorthern Virginia (us-east4)\nOregon (us-west1)\nSeoul, Korea (asia-northeast3)\nSingapore (asia-southeast1)\nTokyo, Japan (asia-northeast1)\n```\n\nYou can follow here if new regions are available: [Gemini API](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/gemini)\n\nYou might want to explicitly set a Google Cloud Project ID, which you can do as follows:\n\n```ruby\n{\n  service: 'vertex-ai-api',\n  project_id: 'PROJECT_ID'\n}\n```\n\n### Custom Version\n\nBy default, the gem uses the `v1` version of the APIs. You may want to use a different version:\n\n```ruby\n# With an API key\nclient = Gemini.new(\n  credentials: {\n    service: 'generative-language-api',\n    api_key: ENV['GOOGLE_API_KEY'],\n    version: 'v1beta'\n  },\n  options: { model: 'gemini-pro', server_sent_events: true }\n)\n\n# With a Service Account Credentials File\nclient = Gemini.new(\n  credentials: {\n    service: 'vertex-ai-api',\n    file_path: 'google-credentials.json',\n    region: 'us-east4',\n    version: 'v1beta'\n  },\n  options: { model: 'gemini-pro', server_sent_events: true }\n)\n\n# With the Service Account Credentials File contents\nclient = Gemini.new(\n  credentials: {\n    service: 'vertex-ai-api',\n    file_contents: File.read('google-credentials.json'),\n    # file_contents: ENV['GOOGLE_CREDENTIALS_FILE_CONTENTS'],\n    region: 'us-east4'\n  },\n  options: { model: 'gemini-pro', server_sent_events: true }\n)\n\n# With Application Default Credentials\nclient = Gemini.new(\n  credentials: {\n    service: 'vertex-ai-api',\n    region: 'us-east4',\n    version: 'v1beta'\n  },\n  options: { model: 'gemini-pro', server_sent_events: true }\n)\n```\n\n## Available Models\n\nThese models are accessible to the repository **author** as of June 2025 in the `us-east4` region. Access to models may vary by region, user, and account. All models here are expected to work, if you can access them. This is just a reference of what a \"typical\" user may expect to have access to right away:\n\n| Model                                    | Vertex AI | Generative Language |\n|------------------------------------------|:---------:|:-------------------:|\n| gemini-pro-vision                        |    ✅     |          🔒         |\n| gemini-pro                               |    ✅     |          ✅         |\n| gemini-1.5-pro-preview-0514              |    ✅     |          🔒         |\n| gemini-1.5-pro-preview-0409              |    ✅     |          🔒         |\n| gemini-1.5-pro                           |    ✅     |          ✅         |\n| gemini-1.5-flash-preview-0514            |    ✅     |          🔒         |\n| gemini-1.5-flash                         |    ✅     |          ✅         |\n| gemini-1.0-pro-vision-latest             |    🔒     |          🔒         |\n| gemini-1.0-pro-vision-001                |    ✅     |          🔒         |\n| gemini-1.0-pro-vision                    |    ✅     |          🔒         |\n| gemini-1.0-pro-latest                    |    🔒     |          ✅         |\n| gemini-1.0-pro-002                       |    ✅     |          🔒         |\n| gemini-1.0-pro-001                       |    ✅     |          ✅         |\n| gemini-1.0-pro                           |    ✅     |          ✅         |\n| gemini-ultra                             |    🔒     |          🔒         |\n| gemini-1.0-ultra                         |    🔒     |          🔒         |\n| gemini-1.0-ultra-001                     |    🔒     |          🔒         |\n| text-embedding-preview-0514              |    🔒     |          🔒         |\n| text-embedding-preview-0409              |    🔒     |          🔒         |\n| text-embedding-004                       |    ✅     |          ✅         |\n| embedding-001                            |    🔒     |          ✅         |\n| text-multilingual-embedding-002          |    ✅     |          🔒         |\n| textembedding-gecko-multilingual@001     |    ✅     |          🔒         |\n| textembedding-gecko-multilingual@latest  |    ✅     |          🔒         |\n| textembedding-gecko@001                  |    ✅     |          🔒         |\n| textembedding-gecko@002                  |    ✅     |          🔒         |\n| textembedding-gecko@003                  |    ✅     |          🔒         |\n| textembedding-gecko@latest               |    ✅     |          🔒         |\n\nYou can follow new models at:\n\n- [Google models](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models)\n  - [Model versions and lifecycle](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/model-versioning)\n\nThis is [the code](https://gist.github.com/gbaptista/d7390901293bce81ee12ff4ec5fed62c) used for generating this table that you can use to explore your own access.\n\n## Usage\n\n### Client\nEnsure that you have all the [required data](#required-data) for authentication.\n\nCreate a new client:\n```ruby\nrequire 'gemini-ai'\n\n# With an API key\nclient = Gemini.new(\n  credentials: {\n    service: 'generative-language-api',\n    api_key: ENV['GOOGLE_API_KEY']\n  },\n  options: { model: 'gemini-pro', server_sent_events: true }\n)\n\n# With a Service Account Credentials File\nclient = Gemini.new(\n  credentials: {\n    service: 'vertex-ai-api',\n    file_path: 'google-credentials.json',\n    region: 'us-east4'\n  },\n  options: { model: 'gemini-pro', server_sent_events: true }\n)\n\n# With the Service Account Credentials File contents\nclient = Gemini.new(\n  credentials: {\n    service: 'vertex-ai-api',\n    file_contents: File.read('google-credentials.json'),\n    # file_contents: ENV['GOOGLE_CREDENTIALS_FILE_CONTENTS'],\n    region: 'us-east4'\n  },\n  options: { model: 'gemini-pro', server_sent_events: true }\n)\n\n# With Application Default Credentials\nclient = Gemini.new(\n  credentials: {\n    service: 'vertex-ai-api',\n    region: 'us-east4'\n  },\n  options: { model: 'gemini-pro', server_sent_events: true }\n)\n```\n\n### Methods\n\n#### Chat\n\n##### stream_generate_content\n\n###### Receiving Stream Events\n\nEnsure that you have enabled [Server-Sent Events](#streaming-vs-server-sent-events-sse) before using blocks for streaming:\n\n```ruby\nclient.stream_generate_content(\n  { contents: { role: 'user', parts: { text: 'hi!' } } }\n) do |event, parsed, raw|\n  puts event\nend\n```\n\nEvent:\n```ruby\n{ 'candidates' =\u003e\n  [{ 'content' =\u003e {\n       'role' =\u003e 'model',\n       'parts' =\u003e [{ 'text' =\u003e 'Hello! How may I assist you?' }]\n     },\n     'finishReason' =\u003e 'STOP',\n     'safetyRatings' =\u003e\n     [{ 'category' =\u003e 'HARM_CATEGORY_HARASSMENT', 'probability' =\u003e 'NEGLIGIBLE' },\n      { 'category' =\u003e 'HARM_CATEGORY_HATE_SPEECH', 'probability' =\u003e 'NEGLIGIBLE' },\n      { 'category' =\u003e 'HARM_CATEGORY_SEXUALLY_EXPLICIT', 'probability' =\u003e 'NEGLIGIBLE' },\n      { 'category' =\u003e 'HARM_CATEGORY_DANGEROUS_CONTENT', 'probability' =\u003e 'NEGLIGIBLE' }] }],\n  'usageMetadata' =\u003e {\n    'promptTokenCount' =\u003e 2,\n    'candidatesTokenCount' =\u003e 8,\n    'totalTokenCount' =\u003e 10\n  } }\n```\n\n###### Without Events\n\nYou can use `stream_generate_content` without events:\n\n```ruby\nresult = client.stream_generate_content(\n  { contents: { role: 'user', parts: { text: 'hi!' } } }\n)\n```\n\nIn this case, the result will be an array with all the received events:\n\n```ruby\n[{ 'candidates' =\u003e\n   [{ 'content' =\u003e {\n        'role' =\u003e 'model',\n        'parts' =\u003e [{ 'text' =\u003e 'Hello! How may I assist you?' }]\n      },\n      'finishReason' =\u003e 'STOP',\n      'safetyRatings' =\u003e\n      [{ 'category' =\u003e 'HARM_CATEGORY_HARASSMENT', 'probability' =\u003e 'NEGLIGIBLE' },\n       { 'category' =\u003e 'HARM_CATEGORY_HATE_SPEECH', 'probability' =\u003e 'NEGLIGIBLE' },\n       { 'category' =\u003e 'HARM_CATEGORY_SEXUALLY_EXPLICIT', 'probability' =\u003e 'NEGLIGIBLE' },\n       { 'category' =\u003e 'HARM_CATEGORY_DANGEROUS_CONTENT', 'probability' =\u003e 'NEGLIGIBLE' }] }],\n   'usageMetadata' =\u003e {\n     'promptTokenCount' =\u003e 2,\n     'candidatesTokenCount' =\u003e 8,\n     'totalTokenCount' =\u003e 10\n   } }]\n```\n\nYou can mix both as well:\n```ruby\nresult = client.stream_generate_content(\n  { contents: { role: 'user', parts: { text: 'hi!' } } }\n) do |event, parsed, raw|\n  puts event\nend\n```\n\n##### generate_content\n\n```ruby\nresult = client.generate_content(\n  { contents: { role: 'user', parts: { text: 'hi!' } } }\n)\n```\n\nResult:\n```ruby\n{ 'candidates' =\u003e\n  [{ 'content' =\u003e { 'parts' =\u003e [{ 'text' =\u003e 'Hello! How can I assist you today?' }], 'role' =\u003e 'model' },\n     'finishReason' =\u003e 'STOP',\n     'index' =\u003e 0,\n     'safetyRatings' =\u003e\n     [{ 'category' =\u003e 'HARM_CATEGORY_SEXUALLY_EXPLICIT', 'probability' =\u003e 'NEGLIGIBLE' },\n      { 'category' =\u003e 'HARM_CATEGORY_HATE_SPEECH', 'probability' =\u003e 'NEGLIGIBLE' },\n      { 'category' =\u003e 'HARM_CATEGORY_HARASSMENT', 'probability' =\u003e 'NEGLIGIBLE' },\n      { 'category' =\u003e 'HARM_CATEGORY_DANGEROUS_CONTENT', 'probability' =\u003e 'NEGLIGIBLE' }] }],\n  'promptFeedback' =\u003e\n  { 'safetyRatings' =\u003e\n    [{ 'category' =\u003e 'HARM_CATEGORY_SEXUALLY_EXPLICIT', 'probability' =\u003e 'NEGLIGIBLE' },\n     { 'category' =\u003e 'HARM_CATEGORY_HATE_SPEECH', 'probability' =\u003e 'NEGLIGIBLE' },\n     { 'category' =\u003e 'HARM_CATEGORY_HARASSMENT', 'probability' =\u003e 'NEGLIGIBLE' },\n     { 'category' =\u003e 'HARM_CATEGORY_DANGEROUS_CONTENT', 'probability' =\u003e 'NEGLIGIBLE' }] } }\n```\n\nAs of the writing of this README, only the `generative-language-api` service supports the `generate_content` method; `vertex-ai-api` does not.\n\n#### Embeddings\n\n##### predict\n\nVertex AI API generates embeddings through the `predict` method ([documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/embeddings/get-text-embeddings)), and you need a client set up to use an embedding model (e.g. `text-embedding-004`):\n\n```ruby\nresult = client.predict(\n  { instances: [{ content: 'What is life?' }],\n    parameters: { autoTruncate: true } }\n)\n```\n\nResult:\n```ruby\n{ 'predictions' =\u003e\n  [{ 'embeddings' =\u003e\n     { 'statistics' =\u003e { 'truncated' =\u003e false, 'token_count' =\u003e 4 },\n       'values' =\u003e\n       [-0.006861076690256596,\n        0.00020840796059928834,\n        -0.028549950569868088,\n        # ...\n        0.0020092015620321035,\n        0.03279878571629524,\n        -0.014905261807143688] } }],\n  'metadata' =\u003e { 'billableCharacterCount' =\u003e 11 } }\n```\n\n##### embed_content\n\nGenerative Language API generates embeddings through the `embed_content` method ([documentation](https://ai.google.dev/api/rest/v1/models/embedContent)), and you need a client set up to use an embedding model (e.g. `text-embedding-004`):\n\n```ruby\nresult = client.embed_content(\n  { content: { parts: [{ text: 'What is life?' }] } }\n)\n```\n\nResult:\n```ruby\n{ 'embedding' =\u003e\n  { 'values' =\u003e\n    [-0.0065307906,\n     -0.0001632607,\n     -0.028370803,\n\n     0.0019950708,\n     0.032798845,\n     -0.014878989] } }\n```\n\n### Modes\n\n#### Text\n\n```ruby\nresult = client.stream_generate_content({\n  contents: { role: 'user', parts: { text: 'hi!' } }\n})\n```\n\nResult:\n```ruby\n[{ 'candidates' =\u003e\n   [{ 'content' =\u003e {\n        'role' =\u003e 'model',\n        'parts' =\u003e [{ 'text' =\u003e 'Hello! How may I assist you?' }]\n      },\n      'finishReason' =\u003e 'STOP',\n      'safetyRatings' =\u003e\n      [{ 'category' =\u003e 'HARM_CATEGORY_HARASSMENT', 'probability' =\u003e 'NEGLIGIBLE' },\n       { 'category' =\u003e 'HARM_CATEGORY_HATE_SPEECH', 'probability' =\u003e 'NEGLIGIBLE' },\n       { 'category' =\u003e 'HARM_CATEGORY_SEXUALLY_EXPLICIT', 'probability' =\u003e 'NEGLIGIBLE' },\n       { 'category' =\u003e 'HARM_CATEGORY_DANGEROUS_CONTENT', 'probability' =\u003e 'NEGLIGIBLE' }] }],\n   'usageMetadata' =\u003e {\n     'promptTokenCount' =\u003e 2,\n     'candidatesTokenCount' =\u003e 8,\n     'totalTokenCount' =\u003e 10\n   } }]\n```\n\n#### Image\n\n![A black and white image of an old piano. The piano is an upright model, with the keys on the right side of the image. The piano is sitting on a tiled floor. There is a small round object on the top of the piano.](https://raw.githubusercontent.com/gbaptista/assets/main/gemini-ai/piano.jpg)\n\n\u003e _Courtesy of [Unsplash](https://unsplash.com/photos/greyscale-photo-of-grand-piano-czPs0z3-Ggg)_\n\nSwitch to the `gemini-pro-vision` model:\n\n```ruby\nclient = Gemini.new(\n  credentials: { service: 'vertex-ai-api', region: 'us-east4' },\n  options: { model: 'gemini-pro-vision', server_sent_events: true }\n)\n```\n\nThen, encode the image as [Base64](https://en.wikipedia.org/wiki/Base64) and add its [MIME type](https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types/Common_types):\n\n```ruby\nrequire 'base64'\n\nresult = client.stream_generate_content(\n  { contents: [\n    { role: 'user', parts: [\n      { text: 'Please describe this image.' },\n      { inline_data: {\n        mime_type: 'image/jpeg',\n        data: Base64.strict_encode64(File.read('piano.jpg'))\n      } }\n    ] }\n  ] }\n)\n```\n\nThe result:\n```ruby\n[{ 'candidates' =\u003e\n   [{ 'content' =\u003e\n      { 'role' =\u003e 'model',\n        'parts' =\u003e\n        [{ 'text' =\u003e\n           ' A black and white image of an old piano. The piano is an upright model, with the keys on the right side of the image. The piano is' }] },\n      'safetyRatings' =\u003e\n      [{ 'category' =\u003e 'HARM_CATEGORY_HARASSMENT', 'probability' =\u003e 'NEGLIGIBLE' },\n       { 'category' =\u003e 'HARM_CATEGORY_HATE_SPEECH', 'probability' =\u003e 'NEGLIGIBLE' },\n       { 'category' =\u003e 'HARM_CATEGORY_SEXUALLY_EXPLICIT', 'probability' =\u003e 'NEGLIGIBLE' },\n       { 'category' =\u003e 'HARM_CATEGORY_DANGEROUS_CONTENT', 'probability' =\u003e 'NEGLIGIBLE' }] }] },\n { 'candidates' =\u003e\n   [{ 'content' =\u003e { 'role' =\u003e 'model', 'parts' =\u003e [{ 'text' =\u003e ' sitting on a tiled floor. There is a small round object on the top of the piano.' }] },\n      'finishReason' =\u003e 'STOP',\n      'safetyRatings' =\u003e\n      [{ 'category' =\u003e 'HARM_CATEGORY_HARASSMENT', 'probability' =\u003e 'NEGLIGIBLE' },\n       { 'category' =\u003e 'HARM_CATEGORY_HATE_SPEECH', 'probability' =\u003e 'NEGLIGIBLE' },\n       { 'category' =\u003e 'HARM_CATEGORY_SEXUALLY_EXPLICIT', 'probability' =\u003e 'NEGLIGIBLE' },\n       { 'category' =\u003e 'HARM_CATEGORY_DANGEROUS_CONTENT', 'probability' =\u003e 'NEGLIGIBLE' }] }],\n   'usageMetadata' =\u003e { 'promptTokenCount' =\u003e 263, 'candidatesTokenCount' =\u003e 50, 'totalTokenCount' =\u003e 313 } }]\n```\n\n#### Video\n\nhttps://gist.github.com/assets/29520/f82bccbf-02d2-4899-9c48-eb8a0a5ef741\n\n\u003e ALT: A white and gold cup is being filled with coffee. The coffee is dark and rich. The cup is sitting on a black surface. The background is blurred.\n\n\u003e _Courtesy of [Pexels](https://www.pexels.com/video/pouring-of-coffee-855391/)_\n\nSwitch to the `gemini-pro-vision` model:\n\n```ruby\nclient = Gemini.new(\n  credentials: { service: 'vertex-ai-api', region: 'us-east4' },\n  options: { model: 'gemini-pro-vision', server_sent_events: true }\n)\n```\n\nThen, encode the video as [Base64](https://en.wikipedia.org/wiki/Base64) and add its [MIME type](https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types/Common_types):\n\n```ruby\nrequire 'base64'\n\nresult = client.stream_generate_content(\n  { contents: [\n    { role: 'user', parts: [\n      { text: 'Please describe this video.' },\n      { inline_data: {\n        mime_type: 'video/mp4',\n        data: Base64.strict_encode64(File.read('coffee.mp4'))\n      } }\n    ] }\n  ] }\n)\n```\n\nThe result:\n```ruby\n[{\"candidates\"=\u003e\n   [{\"content\"=\u003e\n      {\"role\"=\u003e\"model\",\n       \"parts\"=\u003e\n        [{\"text\"=\u003e\n           \" A white and gold cup is being filled with coffee. The coffee is dark and rich. The cup is sitting on a black surface. The background is blurred\"}]},\n     \"safetyRatings\"=\u003e\n      [{\"category\"=\u003e\"HARM_CATEGORY_HARASSMENT\", \"probability\"=\u003e\"NEGLIGIBLE\"},\n       {\"category\"=\u003e\"HARM_CATEGORY_HATE_SPEECH\", \"probability\"=\u003e\"NEGLIGIBLE\"},\n       {\"category\"=\u003e\"HARM_CATEGORY_SEXUALLY_EXPLICIT\", \"probability\"=\u003e\"NEGLIGIBLE\"},\n       {\"category\"=\u003e\"HARM_CATEGORY_DANGEROUS_CONTENT\", \"probability\"=\u003e\"NEGLIGIBLE\"}]}],\n  \"usageMetadata\"=\u003e{\"promptTokenCount\"=\u003e1037, \"candidatesTokenCount\"=\u003e31, \"totalTokenCount\"=\u003e1068}},\n {\"candidates\"=\u003e\n   [{\"content\"=\u003e{\"role\"=\u003e\"model\", \"parts\"=\u003e[{\"text\"=\u003e\".\"}]},\n     \"finishReason\"=\u003e\"STOP\",\n     \"safetyRatings\"=\u003e\n      [{\"category\"=\u003e\"HARM_CATEGORY_HARASSMENT\", \"probability\"=\u003e\"NEGLIGIBLE\"},\n       {\"category\"=\u003e\"HARM_CATEGORY_HATE_SPEECH\", \"probability\"=\u003e\"NEGLIGIBLE\"},\n       {\"category\"=\u003e\"HARM_CATEGORY_SEXUALLY_EXPLICIT\", \"probability\"=\u003e\"NEGLIGIBLE\"},\n       {\"category\"=\u003e\"HARM_CATEGORY_DANGEROUS_CONTENT\", \"probability\"=\u003e\"NEGLIGIBLE\"}]}],\n  \"usageMetadata\"=\u003e{\"promptTokenCount\"=\u003e1037, \"candidatesTokenCount\"=\u003e32, \"totalTokenCount\"=\u003e1069}}]\n```\n\n### Streaming vs. Server-Sent Events (SSE)\n\n[Server-Sent Events (SSE)](https://en.wikipedia.org/wiki/Server-sent_events) is a technology that allows certain endpoints to offer streaming capabilities, such as creating the impression that \"the model is typing along with you,\" rather than delivering the entire answer all at once.\n\nYou can set up the client to use Server-Sent Events (SSE) for all supported endpoints:\n```ruby\nclient = Gemini.new(\n  credentials: { ... },\n  options: { model: 'gemini-pro', server_sent_events: true }\n)\n```\n\nOr, you can decide on a request basis:\n```ruby\nclient.stream_generate_content(\n  { contents: { role: 'user', parts: { text: 'hi!' } } },\n  server_sent_events: true\n)\n```\n\nWith Server-Sent Events (SSE) enabled, you can use a block to receive partial results via events. This feature is particularly useful for methods that offer streaming capabilities, such as `stream_generate_content`:\n\n```ruby\nclient.stream_generate_content(\n  { contents: { role: 'user', parts: { text: 'hi!' } } }\n) do |event, parsed, raw|\n  puts event\nend\n```\n\nEvent:\n```ruby\n{ 'candidates' =\u003e\n  [{ 'content' =\u003e {\n       'role' =\u003e 'model',\n       'parts' =\u003e [{ 'text' =\u003e 'Hello! How may I assist you?' }]\n     },\n     'finishReason' =\u003e 'STOP',\n     'safetyRatings' =\u003e\n     [{ 'category' =\u003e 'HARM_CATEGORY_HARASSMENT', 'probability' =\u003e 'NEGLIGIBLE' },\n      { 'category' =\u003e 'HARM_CATEGORY_HATE_SPEECH', 'probability' =\u003e 'NEGLIGIBLE' },\n      { 'category' =\u003e 'HARM_CATEGORY_SEXUALLY_EXPLICIT', 'probability' =\u003e 'NEGLIGIBLE' },\n      { 'category' =\u003e 'HARM_CATEGORY_DANGEROUS_CONTENT', 'probability' =\u003e 'NEGLIGIBLE' }] }],\n  'usageMetadata' =\u003e {\n    'promptTokenCount' =\u003e 2,\n    'candidatesTokenCount' =\u003e 8,\n    'totalTokenCount' =\u003e 10\n  } }\n```\n\nEven though streaming methods utilize Server-Sent Events (SSE), using this feature doesn't necessarily mean streaming data. For example, when `generate_content` is called with SSE enabled, you will receive all the data at once in a single event, rather than through multiple partial events. This occurs because `generate_content` isn't designed for streaming, even though it is capable of utilizing Server-Sent Events.\n\n#### Server-Sent Events (SSE) Hang\n\nMethod calls will _hang_ until the server-sent events finish, so even without providing a block, you can obtain the final results of the received events:\n\n```ruby\nresult = client.stream_generate_content(\n  { contents: { role: 'user', parts: { text: 'hi!' } } },\n  server_sent_events: true\n)\n```\n\nResult:\n```ruby\n[{ 'candidates' =\u003e\n   [{ 'content' =\u003e {\n        'role' =\u003e 'model',\n        'parts' =\u003e [{ 'text' =\u003e 'Hello! How may I assist you?' }]\n      },\n      'finishReason' =\u003e 'STOP',\n      'safetyRatings' =\u003e\n      [{ 'category' =\u003e 'HARM_CATEGORY_HARASSMENT', 'probability' =\u003e 'NEGLIGIBLE' },\n       { 'category' =\u003e 'HARM_CATEGORY_HATE_SPEECH', 'probability' =\u003e 'NEGLIGIBLE' },\n       { 'category' =\u003e 'HARM_CATEGORY_SEXUALLY_EXPLICIT', 'probability' =\u003e 'NEGLIGIBLE' },\n       { 'category' =\u003e 'HARM_CATEGORY_DANGEROUS_CONTENT', 'probability' =\u003e 'NEGLIGIBLE' }] }],\n   'usageMetadata' =\u003e {\n     'promptTokenCount' =\u003e 2,\n     'candidatesTokenCount' =\u003e 8,\n     'totalTokenCount' =\u003e 10\n   } }]\n```\n\n#### Non-Streaming\n\nDepending on the service, you can use the [`generate_content`](#generate_content) method, which does not stream the answer.\n\nYou can also use methods designed for streaming without necessarily processing partial events; instead, you can wait for the result of all received events:\n\n```ruby\nresult = client.stream_generate_content({\n  contents: { role: 'user', parts: { text: 'hi!' } },\n  server_sent_events: false\n})\n```\n\nResult:\n```ruby\n[{ 'candidates' =\u003e\n   [{ 'content' =\u003e {\n        'role' =\u003e 'model',\n        'parts' =\u003e [{ 'text' =\u003e 'Hello! How may I assist you?' }]\n      },\n      'finishReason' =\u003e 'STOP',\n      'safetyRatings' =\u003e\n      [{ 'category' =\u003e 'HARM_CATEGORY_HARASSMENT', 'probability' =\u003e 'NEGLIGIBLE' },\n       { 'category' =\u003e 'HARM_CATEGORY_HATE_SPEECH', 'probability' =\u003e 'NEGLIGIBLE' },\n       { 'category' =\u003e 'HARM_CATEGORY_SEXUALLY_EXPLICIT', 'probability' =\u003e 'NEGLIGIBLE' },\n       { 'category' =\u003e 'HARM_CATEGORY_DANGEROUS_CONTENT', 'probability' =\u003e 'NEGLIGIBLE' }] }],\n   'usageMetadata' =\u003e {\n     'promptTokenCount' =\u003e 2,\n     'candidatesTokenCount' =\u003e 8,\n     'totalTokenCount' =\u003e 10\n   } }]\n```\n\n### Back-and-Forth Conversations\n\nTo maintain a back-and-forth conversation, you need to append the received responses and build a history for your requests:\n\n```rb\nresult = client.stream_generate_content(\n  { contents: [\n    { role: 'user', parts: { text: 'Hi! My name is Purple.' } },\n    { role: 'model', parts: { text: \"Hello Purple! It's nice to meet you.\" } },\n    { role: 'user', parts: { text: \"What's my name?\" } }\n  ] }\n)\n```\n\nResult:\n```ruby\n[{ 'candidates' =\u003e\n   [{ 'content' =\u003e\n      { 'role' =\u003e 'model',\n        'parts' =\u003e [\n          { 'text' =\u003e \"Purple.\\n\\nYou told me your name was Purple in your first message to me.\\n\\nIs there anything\" }\n        ] },\n      'safetyRatings' =\u003e\n      [{ 'category' =\u003e 'HARM_CATEGORY_HARASSMENT', 'probability' =\u003e 'NEGLIGIBLE' },\n       { 'category' =\u003e 'HARM_CATEGORY_HATE_SPEECH', 'probability' =\u003e 'NEGLIGIBLE' },\n       { 'category' =\u003e 'HARM_CATEGORY_SEXUALLY_EXPLICIT', 'probability' =\u003e 'NEGLIGIBLE' },\n       { 'category' =\u003e 'HARM_CATEGORY_DANGEROUS_CONTENT', 'probability' =\u003e 'NEGLIGIBLE' }] }] },\n { 'candidates' =\u003e\n   [{ 'content' =\u003e { 'role' =\u003e 'model', 'parts' =\u003e [{ 'text' =\u003e ' else I can help you with today, Purple?' }] },\n      'finishReason' =\u003e 'STOP',\n      'safetyRatings' =\u003e\n      [{ 'category' =\u003e 'HARM_CATEGORY_HARASSMENT', 'probability' =\u003e 'NEGLIGIBLE' },\n       { 'category' =\u003e 'HARM_CATEGORY_HATE_SPEECH', 'probability' =\u003e 'NEGLIGIBLE' },\n       { 'category' =\u003e 'HARM_CATEGORY_SEXUALLY_EXPLICIT', 'probability' =\u003e 'NEGLIGIBLE' },\n       { 'category' =\u003e 'HARM_CATEGORY_DANGEROUS_CONTENT', 'probability' =\u003e 'NEGLIGIBLE' }] }],\n   'usageMetadata' =\u003e {\n     'promptTokenCount' =\u003e 24,\n     'candidatesTokenCount' =\u003e 31,\n     'totalTokenCount' =\u003e 55\n   } }]\n```\n\n### Safety Settings\n\nYou can [configure safety attributes](https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/configure-safety-attributes) for your requests.\n\nHarm Categories:\n\u003e `HARM_CATEGORY_UNSPECIFIED`, `HARM_CATEGORY_HARASSMENT`, `HARM_CATEGORY_HATE_SPEECH`, `HARM_CATEGORY_SEXUALLY_EXPLICIT`, `HARM_CATEGORY_DANGEROUS_CONTENT`.\n\nThresholds:\n\u003e `BLOCK_NONE`, `BLOCK_ONLY_HIGH`, `BLOCK_MEDIUM_AND_ABOVE`, `BLOCK_LOW_AND_ABOVE`, `HARM_BLOCK_THRESHOLD_UNSPECIFIED`.\n\nExample:\n```ruby\nclient.stream_generate_content(\n  {\n    contents: { role: 'user', parts: { text: 'hi!' } },\n    safetySettings: [\n      {\n        category: 'HARM_CATEGORY_UNSPECIFIED',\n        threshold: 'BLOCK_ONLY_HIGH'\n      },\n      {\n        category: 'HARM_CATEGORY_HARASSMENT',\n        threshold: 'BLOCK_ONLY_HIGH'\n      },\n      {\n        category: 'HARM_CATEGORY_HATE_SPEECH',\n        threshold: 'BLOCK_ONLY_HIGH'\n      },\n      {\n        category: 'HARM_CATEGORY_SEXUALLY_EXPLICIT',\n        threshold: 'BLOCK_ONLY_HIGH'\n      },\n      {\n        category: 'HARM_CATEGORY_DANGEROUS_CONTENT',\n        threshold: 'BLOCK_ONLY_HIGH'\n      }\n    ]\n  }\n)\n```\n\nGoogle started to block the usage of `BLOCK_NONE` unless:\n\n\u003e _User has requested a restricted HarmBlockThreshold setting BLOCK_NONE. You can get access either (a) through an allowlist via your Google account team, or (b) by switching your account type to monthly invoiced billing via this instruction: https://cloud.google.com/billing/docs/how-to/invoiced-billing_\n\n### System Instructions\n\nSome models support [system instructions](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/prompts/system-instructions):\n\n```ruby\nclient.stream_generate_content(\n  { contents: { role: 'user', parts: { text: 'Hi! Who are you?' } },\n    system_instruction: { role: 'user', parts: { text: 'Your name is Neko.' } } }\n)\n```\n\nOutput:\n```text\nHi! I'm  Neko, a factual language model from Google AI.\n```\n\n```ruby\nclient.stream_generate_content(\n  { contents: { role: 'user', parts: { text: 'Hi! Who are you?' } },\n    system_instruction: {\n      role: 'user', parts: [\n        { text: 'You are a cat.' },\n        { text: 'Your name is Neko.' }\n      ]\n    } }\n)\n```\n\nOutput:\n```text\nMeow! I'm Neko, a fluffy and playful cat. :3\n```\n\n### Counting Tokens\n\nYou can [count tokens](https://ai.google.dev/gemini-api/docs/tokens#count-tokens) and preview how many tokens a request is expected to consume:\n\n```ruby\nclient.count_tokens(\n  { contents: { role: 'user', parts: { text: 'hi!' } } }\n)\n```\n\nOutput for Generative Language API:\n```ruby\n{ 'totalTokens' =\u003e 3 }\n```\n\nOutput for Vertex AI API:\n\n```ruby\n{ 'totalTokens' =\u003e 2, 'totalBillableCharacters' =\u003e 3 }\n```\n\n### JSON Format Responses\n\n\u003e _As of the writing of this README, only the `vertex-ai-api` service and `gemini` models version `1.5` support this feature._\n\nThe Gemini API provides a configuration parameter to [request a response in JSON](https://ai.google.dev/gemini-api/docs/api-overview#json) format:\n\n```ruby\nrequire 'json'\n\nresult = client.stream_generate_content(\n  {\n    contents: {\n      role: 'user',\n      parts: {\n        text: 'List 3 random colors.'\n      }\n    },\n    generation_config: {\n      response_mime_type: 'application/json'\n    }\n\n  }\n)\n\njson_string = result\n              .map { |response| response.dig('candidates', 0, 'content', 'parts') }\n              .map { |parts| parts.map { |part| part['text'] }.join }\n              .join\n\nputs JSON.parse(json_string).inspect\n```\n\nOutput:\n```ruby\n{ 'colors' =\u003e ['Dark Salmon', 'Indigo', 'Lavender'] }\n```\n\n#### JSON Schema\n\n\u003e _While Gemini 1.5 Flash models only accept a text description of the JSON schema you want returned, the Gemini 1.5 Pro models let you pass a schema object (or a Python type equivalent), and the model output will strictly follow that schema. This is also known as controlled generation or constrained decoding._\n\nYou can also provide a [JSON Schema](https://json-schema.org) for the expected JSON output:\n\n```ruby\nrequire 'json'\n\nresult = client.stream_generate_content(\n  {\n    contents: {\n      role: 'user',\n      parts: {\n        text: 'List 3 random colors.'\n      }\n    },\n    generation_config: {\n      response_mime_type: 'application/json',\n      response_schema: {\n        type: 'object',\n        properties: {\n          colors: {\n            type: 'array',\n            items: {\n              type: 'object',\n              properties: {\n                name: {\n                  type: 'string'\n                }\n              }\n            }\n          }\n        }\n      }\n    }\n  }\n)\n\njson_string = result\n              .map { |response| response.dig('candidates', 0, 'content', 'parts') }\n              .map { |parts| parts.map { |part| part['text'] }.join }\n              .join\n\nputs JSON.parse(json_string).inspect\n```\n\nOutput:\n\n```ruby\n{ 'colors' =\u003e [\n  { 'name' =\u003e 'Lavender Blush' },\n  { 'name' =\u003e 'Medium Turquoise' },\n  { 'name' =\u003e 'Dark Slate Gray' }\n] }\n```\n\n#### Models That Support JSON\n\nThese models are accessible to the repository **author** as of June 2025 in the `us-east4` region. Access to models may vary by region, user, and account.\n\n- ❌ Does not support JSON mode.\n- 🟡 Supports JSON mode but not Schema.\n- ✅ Supports JSON mode and Schema.\n- 🔒 I don't have access to the model.\n\n| Model                                    | Vertex AI | Generative Language |\n|------------------------------------------|:---------:|:-------------------:|\n| gemini-pro-vision                        |    ❌     |          🔒         |\n| gemini-pro                               |    🟡     |          ❌         |\n| gemini-1.5-pro-preview-0514              |    ✅     |          🔒         |\n| gemini-1.5-pro-preview-0409              |    ✅     |          🔒         |\n| gemini-1.5-pro                           |    ✅     |          ❌         |\n| gemini-1.5-flash-preview-0514            |    🟡     |          🔒         |\n| gemini-1.5-flash                         |    🟡     |          ❌         |\n| gemini-1.0-pro-vision-latest             |    🔒     |          🔒         |\n| gemini-1.0-pro-vision-001                |    ❌     |          🔒         |\n| gemini-1.0-pro-vision                    |    ❌     |          🔒         |\n| gemini-1.0-pro-latest                    |    🔒     |          ❌         |\n| gemini-1.0-pro-002                       |    🟡     |          🔒         |\n| gemini-1.0-pro-001                       |    ❌     |          ❌         |\n| gemini-1.0-pro                           |    🟡     |          ❌         |\n| gemini-ultra                             |    🔒     |          🔒         |\n| gemini-1.0-ultra                         |    🔒     |          🔒         |\n| gemini-1.0-ultra-001                     |    🔒     |          🔒         |\n\n\n### Tools (Functions) Calling\n\n\u003e As of the writing of this README, only the `vertex-ai-api` service and the `gemini-pro` model [supports](https://cloud.google.com/vertex-ai/docs/generative-ai/multimodal/function-calling#supported_models) tools (functions) calls.\n\nYou can provide specifications for [tools (functions)](https://cloud.google.com/vertex-ai/docs/generative-ai/multimodal/function-calling) using [JSON Schema](https://json-schema.org) to generate potential calls to them:\n\n```ruby\ninput = {\n  tools: {\n    function_declarations: [\n      {\n        name: 'date_and_time',\n        description: 'Returns the current date and time in the ISO 8601 format for a given timezone.',\n        parameters: {\n          type: 'object',\n          properties: {\n            timezone: {\n              type: 'string',\n              description: 'A string represents the timezone to be used for providing a datetime, following the IANA (Internet Assigned Numbers Authority) Time Zone Database. Examples include \"Asia/Tokyo\" and \"Europe/Paris\". If not provided, the default timezone is the user\\'s current timezone.'\n            }\n          }\n        }\n      }\n    ]\n  },\n  contents: [\n    { role: 'user', parts: { text: 'What time is it?' } }\n  ]\n}\n\nresult = client.stream_generate_content(input)\n```\n\nWhich may return a request to perform a call:\n```ruby\n[{ 'candidates' =\u003e\n   [{ 'content' =\u003e {\n        'role' =\u003e 'model',\n        'parts' =\u003e [{ 'functionCall' =\u003e {\n          'name' =\u003e 'date_and_time',\n          'args' =\u003e { 'timezone' =\u003e 'local' }\n        } }]\n      },\n      'finishReason' =\u003e 'STOP',\n      'safetyRatings' =\u003e\n      [{ 'category' =\u003e 'HARM_CATEGORY_HARASSMENT', 'probability' =\u003e 'NEGLIGIBLE' },\n       { 'category' =\u003e 'HARM_CATEGORY_HATE_SPEECH', 'probability' =\u003e 'NEGLIGIBLE' },\n       { 'category' =\u003e 'HARM_CATEGORY_SEXUALLY_EXPLICIT', 'probability' =\u003e 'NEGLIGIBLE' },\n       { 'category' =\u003e 'HARM_CATEGORY_DANGEROUS_CONTENT', 'probability' =\u003e 'NEGLIGIBLE' }] }],\n   'usageMetadata' =\u003e { 'promptTokenCount' =\u003e 5, 'totalTokenCount' =\u003e 5 } }]\n```\n\nBased on these results, you can perform the requested calls and provide their outputs:\n```ruby\ngem 'tzinfo', '~\u003e 2.0', '\u003e= 2.0.6'\n```\n\n```ruby\nrequire 'tzinfo'\nrequire 'time'\n\nfunction_calls = result.dig(0, 'candidates', 0, 'content', 'parts').filter do |part|\n  part.key?('functionCall')\nend\n\nfunction_parts = []\n\nfunction_calls.each do |function_call|\n  next unless function_call['functionCall']['name'] == 'date_and_time'\n\n  timezone = function_call.dig('functionCall', 'args', 'timezone')\n\n  time = if !timezone.nil? \u0026\u0026 timezone != '' \u0026\u0026 timezone.downcase != 'local'\n           TZInfo::Timezone.get(timezone).now\n         else\n           Time.now\n         end\n\n  function_output = time.iso8601\n\n  function_parts \u003c\u003c {\n    functionResponse: {\n      name: function_call['functionCall']['name'],\n      response: {\n        name: function_call['functionCall']['name'],\n        content: function_output\n      }\n    }\n  }\nend\n\ninput[:contents] \u003c\u003c result.dig(0, 'candidates', 0, 'content')\ninput[:contents] \u003c\u003c { role: 'function', parts: function_parts }\n```\n\nThis will be equivalent to the following final input:\n```ruby\n{ tools: { function_declarations: [\n  { name: 'date_and_time',\n    description: 'Returns the current date and time in the ISO 8601 format for a given timezone.',\n    parameters: {\n      type: 'object',\n      properties: {\n        timezone: {\n          type: 'string',\n          description: \"A string represents the timezone to be used for providing a datetime, following the IANA (Internet Assigned Numbers Authority) Time Zone Database. Examples include \\\"Asia/Tokyo\\\" and \\\"Europe/Paris\\\". If not provided, the default timezone is the user's current timezone.\"\n        }\n      }\n    } }\n] },\n  contents: [\n    { role: 'user', parts: { text: 'What time is it?' } },\n    { role: 'model',\n      parts: [\n        { functionCall: { name: 'date_and_time', args: { timezone: 'local' } } }\n      ] },\n    { role: 'function',\n      parts: [{ functionResponse: {\n        name: 'date_and_time',\n        response: {\n          name: 'date_and_time',\n          content: '2023-12-13T21:15:11-03:00'\n        }\n      } }] }\n  ] }\n```\n\nWith the input properly arranged, you can make another request:\n```ruby\nresult = client.stream_generate_content(input)\n```\n\nWhich will result in:\n```ruby\n[{ 'candidates' =\u003e\n   [{ 'content' =\u003e { 'role' =\u003e 'model', 'parts' =\u003e [{ 'text' =\u003e 'It is 21:15.' }] },\n      'finishReason' =\u003e 'STOP',\n      'safetyRatings' =\u003e\n      [{ 'category' =\u003e 'HARM_CATEGORY_HARASSMENT', 'probability' =\u003e 'NEGLIGIBLE' },\n       { 'category' =\u003e 'HARM_CATEGORY_HATE_SPEECH', 'probability' =\u003e 'NEGLIGIBLE' },\n       { 'category' =\u003e 'HARM_CATEGORY_SEXUALLY_EXPLICIT', 'probability' =\u003e 'NEGLIGIBLE' },\n       { 'category' =\u003e 'HARM_CATEGORY_DANGEROUS_CONTENT', 'probability' =\u003e 'NEGLIGIBLE' }] }],\n   'usageMetadata' =\u003e { 'promptTokenCount' =\u003e 5, 'candidatesTokenCount' =\u003e 9, 'totalTokenCount' =\u003e 14 } }]\n```\n\n### New Functionalities and APIs\n\nGoogle may launch a new endpoint that we haven't covered in the Gem yet. If that's the case, you may still be able to use it through the `request` method. For example, `stream_generate_content` is just a wrapper for `models/gemini-pro:streamGenerateContent` (Generative Language API) or `publishers/google/models/gemini-pro:streamGenerateContent` (Vertex AI API), which you can call directly like this:\n\n```ruby\n# Generative Language API\nresult = client.request(\n  'models/gemini-pro:streamGenerateContent',\n  { contents: { role: 'user', parts: { text: 'hi!' } } },\n  request_method: 'POST',\n  server_sent_events: true\n)\n```\n\n```ruby\n# Vertex AI API\nresult = client.request(\n  'publishers/google/models/gemini-pro:streamGenerateContent',\n  { contents: { role: 'user', parts: { text: 'hi!' } } },\n  request_method: 'POST',\n  server_sent_events: true\n)\n```\n\n### Request Options\n\n#### Adapter\n\nTo enable streaming, the gem uses [Faraday](https://github.com/lostisland/faraday) with the [Typhoeus](https://github.com/typhoeus/typhoeus) adapter by default.\n\nYou can use a different adapter if you want:\n\n```ruby\nrequire 'faraday/net_http'\n\nclient = Gemini.new(\n  credentials: { service: 'vertex-ai-api', region: 'us-east4' },\n  options: {\n    model: 'gemini-pro',\n    connection: { adapter: :net_http }\n  }\n)\n```\n\n#### Timeout\n\nYou can set the maximum number of seconds to wait for the request to complete with the `timeout` option:\n\n```ruby\nclient = Gemini.new(\n  credentials: { service: 'vertex-ai-api', region: 'us-east4' },\n  options: {\n    model: 'gemini-pro',\n    connection: { request: { timeout: 5 } }\n  }\n)\n```\n\nYou can also have more fine-grained control over [Faraday's Request Options](https://lostisland.github.io/faraday/#/customization/request-options?id=request-options) if you prefer:\n\n```ruby\nclient = Gemini.new(\n  credentials: { service: 'vertex-ai-api', region: 'us-east4' },\n  options: {\n    model: 'gemini-pro',\n    connection: {\n      request: {\n        timeout: 5,\n        open_timeout: 5,\n        read_timeout: 5,\n        write_timeout: 5\n      }\n    }\n  }\n)\n```\n\n\n### Error Handling\n\n#### Rescuing\n\n```ruby\nrequire 'gemini-ai'\n\nbegin\n  client.stream_generate_content({\n    contents: { role: 'user', parts: { text: 'hi!' } }\n  })\nrescue Gemini::Errors::GeminiError =\u003e error\n  puts error.class # Gemini::Errors::RequestError\n  puts error.message # 'the server responded with status 500'\n\n  puts error.payload\n  # { contents: [{ role: 'user', parts: { text: 'hi!' } }],\n  #   generationConfig: { candidateCount: 1 },\n  #   ...\n  # }\n\n  puts error.request\n  # #\u003cFaraday::ServerError response={:status=\u003e500, :headers...\nend\n```\n\n#### For Short\n\n```ruby\nrequire 'gemini-ai/errors'\n\nbegin\n  client.stream_generate_content({\n    contents: { role: 'user', parts: { text: 'hi!' } }\n  })\nrescue GeminiError =\u003e error\n  puts error.class # Gemini::Errors::RequestError\nend\n```\n\n#### Errors\n\n```ruby\nGeminiError\n\nMissingProjectIdError\nUnsupportedServiceError\nConflictingCredentialsError\nBlockWithoutServerSentEventsError\n\nRequestError\n```\n\n## Development\n\n```bash\nbundle\nrubocop -A\n\nrspec\n\nbundle exec ruby spec/tasks/run-available-models.rb\nbundle exec ruby spec/tasks/run-embed.rb\nbundle exec ruby spec/tasks/run-generate.rb\nbundle exec ruby spec/tasks/run-json.rb\nbundle exec ruby spec/tasks/run-safety.rb\nbundle exec ruby spec/tasks/run-system.rb\n```\n\n### Purpose\n\nThis Gem is designed to provide low-level access to Gemini, enabling people to build abstractions on top of it. If you are interested in more high-level abstractions or more user-friendly tools, you may want to consider [Nano Bots](https://github.com/icebaker/ruby-nano-bots) 💎 🤖.\n\n### Publish to RubyGems\n\n```bash\ngem build gemini-ai.gemspec\n\ngem signin\n\ngem push gemini-ai-4.2.0.gem\n```\n\n### Updating the README\n\nInstall [Babashka](https://babashka.org):\n\n```sh\ncurl -s https://raw.githubusercontent.com/babashka/babashka/master/install | sudo bash\n```\n\nUpdate the `template.md` file and then:\n\n```sh\nbb tasks/generate-readme.clj\n```\n\nTrick for automatically updating the `README.md` when `template.md` changes:\n\n```sh\nsudo pacman -S inotify-tools # Arch / Manjaro\nsudo apt-get install inotify-tools # Debian / Ubuntu / Raspberry Pi OS\nsudo dnf install inotify-tools # Fedora / CentOS / RHEL\n\nwhile inotifywait -e modify template.md; do bb tasks/generate-readme.clj; done\n```\n\nTrick for Markdown Live Preview:\n```sh\npip install -U markdown_live_preview\n\nmlp README.md -p 8076\n```\n\n## Resources and References\n\nThese resources and references may be useful throughout your learning process.\n\n- [Google AI for Developers](https://ai.google.dev)\n- [Get started with the Gemini API ](https://ai.google.dev/docs)\n- [Getting Started with the Vertex AI Gemini API with cURL](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/getting-started/intro_gemini_curl.ipynb)\n- [Gemini API Documentation](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/gemini)\n- [Vertex AI API Documentation](https://cloud.google.com/vertex-ai/docs/reference)\n  - [REST Documentation](https://cloud.google.com/vertex-ai/docs/reference/rest)\n  - [Get text embeddings](https://cloud.google.com/vertex-ai/generative-ai/docs/embeddings/get-text-embeddings)\n  - [Use system instructions](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/prompts/system-instructions)\n  - [Configure safety attributes](https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/configure-safety-attributes)\n- [Google models](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models)\n  - [Model versions and lifecycle](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/model-versioning)\n- [Google DeepMind Gemini](https://deepmind.google/technologies/gemini/)\n- [Stream responses from Generative AI models](https://cloud.google.com/vertex-ai/docs/generative-ai/learn/streaming)\n- [Function calling](https://cloud.google.com/vertex-ai/docs/generative-ai/multimodal/function-calling)\n\n## Disclaimer\n\nThis is not an official Google project, nor is it affiliated with Google in any way.\n\nThis software is distributed under the [MIT License](https://github.com/gbaptista/gemini-ai/blob/main/LICENSE). This license includes a disclaimer of warranty. Moreover, the authors assume no responsibility for any damage or costs that may result from using this project. Use the Gemini AI Ruby Gem at your own risk.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgbaptista%2Fgemini-ai","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgbaptista%2Fgemini-ai","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgbaptista%2Fgemini-ai/lists"}