{"id":20600959,"url":"https://github.com/maxidonkey/delphigroqcloud","last_synced_at":"2025-09-26T09:30:34.057Z","repository":{"id":262911309,"uuid":"888085146","full_name":"MaxiDonkey/DelphiGroqCloud","owner":"MaxiDonkey","description":"The GroqCloud API wrapper for Delphi provides access to models from Meta, OpenAI, MistralAI and Google on Groq’s LPUs, offering chat, text generation, image analysis, audio transcription, JSON output, tool integration, and content moderation capabilities.","archived":false,"fork":false,"pushed_at":"2024-12-25T07:21:07.000Z","size":230,"stargazers_count":14,"open_issues_count":0,"forks_count":2,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-01-17T01:45:43.561Z","etag":null,"topics":["api-wrapper","audio-transcription","delphi","gpt","groq","groq-ai","groq-api","image-prompting","llama","llama3-1","llama3-2","lpu","vision","vision-api"],"latest_commit_sha":null,"homepage":"","language":"Pascal","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MaxiDonkey.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-11-13T19:39:27.000Z","updated_at":"2025-01-15T13:07:21.000Z","dependencies_parsed_at":"2024-11-15T01:18:31.920Z","dependency_job_id":"2ae682fa-574c-443f-8004-d7470f1c788c","html_url":"https://github.com/MaxiDonkey/DelphiGroqCloud","commit_stats":null,"previous_names":["maxidonkey/delphigroqcloud"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MaxiDonkey%2FDelphiGroqCloud","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MaxiDonkey%2FDelphiGroqCloud/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MaxiDonkey%2FDelphiGroqCloud/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MaxiDonkey%2FDelphiGroqCloud/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MaxiDonkey","download_url":"https://codeload.github.com/MaxiDonkey/DelphiGroqCloud/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":234300371,"owners_count":18810607,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api-wrapper","audio-transcription","delphi","gpt","groq","groq-ai","groq-api","image-prompting","llama","llama3-1","llama3-2","lpu","vision","vision-api"],"created_at":"2024-11-16T09:07:38.728Z","updated_at":"2025-09-26T09:30:28.750Z","avatar_url":"https://github.com/MaxiDonkey.png","language":"Pascal","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Delphi GroqCloud API\n\n___\n![GitHub](https://img.shields.io/badge/IDE%20Version-Delphi%2010.3/11/12-yellow)\n![GitHub](https://img.shields.io/badge/platform-all%20platforms-green)\n![GitHub](https://img.shields.io/badge/Updated%20the%2011/13/2024-blue)\n\n\u003cbr/\u003e\n\u003cbr/\u003e\n\n- [Introduction](#Introduction)\n- [Groq cloud console](#Groq-cloud-console)\n    - [Get a key](#Get-a-key)\n    - [Settings](#Settings)\n- [Usage](#Usage)\n    - [Asynchronous callback mode management](#Asynchronous-callback-mode-management)\n    - [Groq models overview](#Groq-models-overview)\n    - [Embeddings](#Embeddings)\n    - [Text generation](#Text-generation)\n        - [Chat completion](#Chat-completion)\n             - [Synchronously text generation example](#Synchronously-text-generation-example)\n             - [Asynchronously text generation example](#Asynchronously-text-generation-example)\n        - [Stream chat](#Stream-chat)\n             - [Synchronously chat stream](#Synchronously-chat-stream)\n             - [Asynchronously chat stream](#Asynchronously-chat-stream) \n        - [Build an interactive chat](#Build-an-interactive-chat)\n        - [System instructions](#System-instructions)\n        - [Configure text generation](#Configure-text-generation)\n    - [Vision](#Vision)\n        - [Supported Model](#Supported-Model)\n        - [Supported image MIME](#Supported-image-MIME)\n        - [How to use vision](#How-to-use-vision)\n            - [Asynchronous vision using a base64-encoded image](#Asynchronous-vision-using-a-base64-encoded-image)\n            - [Asynchronous vision using an image URL](#Asynchronous-vision-using-an-image-URL)\n            - [JSON Mode with Images](#JSON-Mode-with-Images)\n            - [Limitations](#Limitations)\n    - [Speech](#Speech)\n        - [Supported models](#Supported-models) \n        - [Transcription code example](#Transcription-code-example)\n        - [Translation code example](#Translation-code-example)\n    - [Tool use](#Tool-use)\n        - [How tool use works](#How-tool-use-works)\n        - [Supported models](#Supported-models)\n        - [Tool use code example](#Tool-use-code-example)\n        - [How to create a tool](#How-to-create-a-tool)\n    - [Content moderation](#Content-moderation)\n    - [Fine-tuning](#Fine-tuning)\n    - [Display methods for the tutorial ](#Display-methods-for-the-tutorial )\n- [Contributing](#contributing)\n- [License](#license)\n\n\u003cbr/\u003e\n\u003cbr/\u003e\n\n# Introduction\n\nWelcome to the unofficial **GroqCloud API Wrapper** for **Delphi**. This project provides a **Delphi** interface for accessing and interacting with the powerful language models available on **GroqCloud**, including those developed by : \u003cbr/\u003e\n      **`Meta`** \u003csub\u003eLLama\u003c/sub\u003e, **`OpenAI`** \u003csub\u003eWhisper\u003c/sub\u003e, **`MistralAI`** \u003csub\u003emixtral\u003c/sub\u003e, and **`Google`** \u003csub\u003eGemma\u003c/sub\u003e. \u003cbr/\u003e With this library, you can seamlessly integrate state-of-the-art language generation, chat and vision capabilities, code generation, or speech-to-text transcription into your **Delphi** applications.\n\n**GroqCloud** offers a high-performance, efficient platform optimized for running large language models via its proprietary Language Processing Units (LPUs), delivering speed and energy efficiency that surpass traditional GPUs. This wrapper simplifies access to these models, allowing you to leverage **GroqCloud's** cutting-edge infrastructure without the overhead of managing the underlying hardware.\n\nFor more details on GroqCloud's offerings, visit the [official GroqCloud documentation](https://groq.com/groqcloud/).\n\n\u003cbr/\u003e\n\n# Groq cloud console\n\n## Get a key\n\nTo initialize the API instance, you need to obtain an [API key](https://console.groq.com/keys) from GroqCloud.\n\nOnce you have a token, you can initialize `IGroq` interface, which is an entry point to the API.\n\nDue to the fact that there can be many parameters and not all of them are required, they are configured using an anonymous function.\n\n\u003e [!NOTE]\n\u003e```Pascal\n\u003euses Groq;\n\u003e\n\u003evar GroqCloud := TGroqFactory.CreateInstance(API_KEY);\n\u003e```\n\n\u003e[!Warning]\n\u003e To use the examples provided in this tutorial, especially to work with asynchronous methods, I recommend defining the Groq interface with the widest possible scope.\n\u003e\u003cbr/\u003e\n\u003e So, set `GroqCloud := TGroqFactory.CreateInstance(API_KEY);` in the `OnCreate` event of your application.\n\u003e\u003cbr/\u003e \n\u003eWhere `GroqCloud: IGroq`\n\n\u003cbr/\u003e\n\n## Settings\n\nYou can access your GroqCloud account settings to view your payment information, usage, limits, logs, teams, and profile by following [this link](https://console.groq.com/settings).\n\n\u003cbr/\u003e\n\n# Usage\n\n## Asynchronous callback mode management\n\nIn the context of asynchronous methods, for a method that does not involve streaming, callbacks use the following generic record: `TAsynCallBack\u003cT\u003e = record` defined in the `Gemini.Async.Support.pas` unit. This record exposes the following properties:\n\n```Pascal\n   TAsynCallBack\u003cT\u003e = record\n   ... \n       Sender: TObject;\n       OnStart: TProc\u003cTObject\u003e;\n       OnSuccess: TProc\u003cTObject, T\u003e;\n       OnError: TProc\u003cTObject, string\u003e; \n```\n\u003cbr/\u003e\n\nFor methods requiring streaming, callbacks use the generic record `TAsynStreamCallBack\u003cT\u003e = record`, also defined in the `Gemini.Async.Support.pas` unit. This record exposes the following properties:\n\n```Pascal\n   TAsynCallBack\u003cT\u003e = record\n   ... \n       Sender: TObject;\n       OnStart: TProc\u003cTObject\u003e;\n       OnSuccess: TProc\u003cTObject, T\u003e;\n       OnProgress: TProc\u003cTObject, T\u003e;\n       OnError: TProc\u003cTObject, string\u003e;\n       OnCancellation: TProc\u003cTObject\u003e;\n       OnDoCancel: TFunc\u003cBoolean\u003e;\n```\n\nThe name of each property is self-explanatory; if needed, refer to the internal documentation for more details.\n\n\u003cbr/\u003e\n\n## Groq models overview\n\nGroqCloud currently supports the [following models](https://console.groq.com/docs/models).\n\nHosted models can be accessed directly via the GroqCloud Models API endpoint by using the model IDs listed above. To retrieve a JSON list of all available models, use the endpoint at `https://api.groq.com/openai/v1/models`.\n\n1. **Synchronously**\n\n```Pascal\n// uses Groq, Groq.Models;\n\n  var Models := GroqCloud.Models.List;\n  try\n    for var Item in Models.Data do\n      WriteLn(Item.Id);\n  finally\n    Models.Free;\n  end;\n```\n\n2. **Asynchronously**\n\n```Pascal\n// uses Groq, Groq.Models;\n\n  GroqCloud.Models.AsynList(\n    function : TAsynModels\n    begin\n      Result.Sender := Memo1; //Set a TMemo on the form\n      Result.OnSuccess :=\n         procedure (Sender: TObject; Models: TModels)\n         begin\n           var M := Sender as TMemo;\n           for var Item in Models.Data do\n             begin\n               M.Lines.Text := M.Text + Item.Id + sLineBreak;\n               M.Perform(WM_VSCROLL, SB_BOTTOM, 0);\n             end;\n         end;\n      Result.OnError :=\n        procedure (Sender: TObject; Error: string)\n        begin\n          var M := Sender as TMemo;\n          M.Lines.Text := M.Text + Error + sLineBreak;\n          M.Perform(WM_VSCROLL, SB_BOTTOM, 0);\n        end;\n    end);\n```\n\n\u003cbr/\u003e\n\n## Embeddings\n\n**GroqCloud** does not provide any solutions for text integration.\n\n\u003cbr/\u003e\n\n## Text generation\n\n### Chat completion\n\nThe **Groq Chat Completions API** interprets a series of messages and produces corresponding response outputs. These models can handle either multi-turn conversations or single-interaction tasks.\n\nJSON Mode (Beta) JSON mode is currently in beta and ensures that all chat completions are in valid JSON format.\n\n**How to Use:** \u003cbr/\u003e\n  1. Include `\"response_format\": {\"type\": \"json_object\"}` in your chat completion request.\n  2. In the system prompt, specify the structure of the desired JSON output (see sample system prompts below).\n\u003cbr/\u003e\n\n**Best Practices for Optimal Beta Performance:** \u003cbr/\u003e\n- For JSON generation, Mixtral is the most effective model, followed by Gemma, and then Llama.\n- Use pretty-printed JSON for better readability over compact JSON.\n- Keep prompts as concise as possible.\n\u003cbr/\u003e\n\n**Beta Limitations:** \u003cbr/\u003e\n- Streaming is not supported.\n- Stop sequences are not supported.\n\u003cbr/\u003e\n\n**Error Code:** \u003cbr/\u003e\nIf JSON generation fails, `Groq` will respond with a **400 error**, specifying `json_validate_failed` as the error code.\n\n\u003cbr/\u003e\n\n\u003e[!NOTE]\n\u003e We will use only Meta models in all the examples provided for text generation.\n\u003e\n\n\u003cbr/\u003e\n\n#### Synchronously text generation example\n\nThe `GroqCloud` API allows for text generation using various inputs, like text and images. It's versatile and can support a wide array of applications, including: \u003cbr/\u003e\n\n- Creative writing\n- Text completion\n- Summarizing open-ended text\n- Chatbot development\n- Any custom use cases you have in mind\n\nIn the examples below, we'll use the `Display` procedures to make things simpler.\n\u003e[!TIP]\n\u003e```Pascal\n\u003eprocedure Display(Sender: TObject; Value: string); overload;\n\u003ebegin\n\u003e  var M := Sender as TMemo;\n\u003e  M.Lines.Text := M.Text + Value + sLineBreak;\n\u003e  M.Perform(WM_VSCROLL, SB_BOTTOM, 0);\n\u003eend;\n\u003e```\n\u003e\n\u003e```Pascal\n\u003eprocedure Display(Sender: TObject; Chat: TChat); overload;\n\u003ebegin\n\u003e  for var Choice in Chat.Choices do\n\u003e    Display(Sender, Choice.Message.Content);\n\u003eend;\n\u003e```\n\n\n```Pascal\n// uses Groq, Groq.Chat;\n\n  var Chat := GroqCloud.Chat.Create(\n    procedure (Params: TChatParams)\n    begin\n      Params.Messages([TPayload.User('Explain the importance of fast language models')]);\n      Params.Model('llama-3.1-8b-instant');\n    end);\n  //Set a TMemo on the form\n  try\n    Display(Memo1, Chat);\n  finally\n    Chat.Free;\n  end;\n```\n\n\u003cbr/\u003e\n\n#### Asynchronously text generation example\n\n```Pascal\n// uses Groq, Groq.Chat;\n\n  GroqCloud.Chat.AsynCreate(\n    procedure (Params: TChatParams)\n    begin\n      Params.Messages([TPayload.User('Explain the importance of fast language models')]);\n      Params.Model('llama-3.1-70b-versatile');\n    end,\n    //Set a TMemo on the form\n    function : TAsynChat\n    begin\n      Result.Sender := Memo1;\n      Result.OnSuccess := Display;\n      Result.OnError := Display;\n    end);\n```\n\n\u003cbr/\u003e\n\n### Stream chat\n\n#### Synchronously chat stream\n\nIn the examples below, we'll use the `Display` procedures to make things simpler.\n\u003e[!TIP]\n\u003e```Pascal\n\u003eprocedure DisplayStream(Sender: TObject; Value: string); overload;\n\u003ebegin\n\u003e  var M := Sender as TMemo;\n\u003e  for var index := 1 to Value.Length  do\n\u003e    if Value.Substring(index).StartsWith(#13)\n\u003e      then\n\u003e        begin\n\u003e          M.Lines.Text := M.Text + sLineBreak;\n\u003e          M.Perform(WM_VSCROLL, SB_BOTTOM, 0);\n\u003e        end\n\u003e      else\n\u003e        begin\n\u003e          M.Lines.BeginUpdate;\n\u003e          try\n\u003e            M.Lines.Text := M.Text + Value[index];\n\u003e            M.Perform(WM_VSCROLL, SB_BOTTOM, 0);\n\u003e          finally\n\u003e            M.Lines.EndUpdate;\n\u003e          end;\n\u003e        end;\n\u003eend;\n\u003e```\n\u003e\n\u003e```Pascal\n\u003eprocedure DisplayStream(Sender: TObject; Chat: TChat); overload;\n\u003ebegin\n\u003e  for var Item in Chat.Choices do\n\u003e    if Assigned(Item.Delta) then\n\u003e      DisplayStream(Sender, Item.Delta.Content)\n\u003e    else\n\u003e    if Assigned(Item.Message) then\n\u003e      DisplayStream(Sender, Item.Message.Content);\n\u003eend;\n\u003e```\n\n```Pascal\n// uses Groq, Groq.Chat;\n\n  GroqCloud.Chat.CreateStream(\n    procedure (Params: TChatParams)\n    begin\n      Params.Messages([TPayload.User('How did we come to develop thermodynamics?')]);\n      Params.Model('llama3-70b-8192');\n      Params.Stream(True);\n    end,\n    procedure (var Chat: TChat; IsDone: Boolean; var Cancel: Boolean)\n    begin\n      if Assigned(Chat) then\n        DisplayStream(Memo1, Chat);\n    end);\n```\n\n\u003cbr/\u003e\n\n#### Asynchronously chat stream\n\n```Pascal\n// uses Groq, Groq.Chat;\n\n  GroqCloud.Chat.AsynCreateStream(\n    procedure (Params: TChatParams)\n    begin\n      Params.Messages([TPayload.User('How did we come to develop thermodynamics?')]);\n      Params.Model('llama-3.1-70b-versatile');\n      Params.Stream(True);\n    end,\n    function : TAsynChatStream\n    begin\n      Result.Sender := Memo1;\n      Result.OnProgress := DisplayStream;\n      Result.OnError := DisplayStream;\n    end);\n```\n\n\u003cbr/\u003e\n\n### Build an interactive chat\n\nYou can utilize the `GroqCloud` API to build interactive chat experiences customized for your users. With the API’s chat capability, you can facilitate multiple rounds of questions and answers, allowing users to gradually work toward their solutions or get support for complex, multi-step issues. This feature is particularly valuable for applications that need ongoing interaction, like :\n- Chatbots, \n- Educational tools\n- Customer support assistants.\n\nHere’s an asynchrounly sample of a simple chat setup:\n\n```Pascal\n// uses Groq, Groq.Chat;\n\n  GroqCloud.Chat.AsynCreateStream(\n    procedure (Params: TChatParams)\n    begin\n      Params.Model('llama-3.2-3b-preview');\n      Params.Messages([\n        TPayload.User('Hello'),\n        TPayload.Assistant('Great to meet you. What would you like to know?'),\n        TPayload.User('I have two dogs in my house. How many paws are in my house?')\n      ]);\n      Params.Stream(True);\n    end,\n    //Set a TMemo on the form\n    function : TAsynChatStream\n    begin\n      Result.Sender := Memo1;\n      Result.OnProgress := DisplayStream;\n      Result.OnError := DisplayStream;\n    end);\n```\n\u003cbr/\u003e\n\n### System instructions\n\nWhen configuring an AI model, you have the option to set guidelines for how it should respond. For instance, you could assign it a particular role, like  `act as a mathematician` or give it instructions on tone, such as `peak like a military instructor`. These guidelines are established by setting up system instructions when the model is initialized.\n\nSystem instructions allow you to customize the model’s behavior to suit specific needs and use cases. Once configured, they add context that helps guide the model to perform tasks more accurately according to predefined guidelines throughout the entire interaction. These instructions apply across multiple interactions with the model.\n\nSystem instructions can be used for several purposes, such as:\n\n- **Defining a persona or role (e.g., configuring the model to function as a customer service chatbot)**\n- **Specifying an output format (like Markdown, JSON, or YAML)**\n- **Adjusting the output style and tone (such as modifying verbosity, formality, or reading level)**\n- **Setting goals or rules for the task (for example, providing only a code snippet without additional explanation)**\n- **Supplying relevant context (like a knowledge cutoff date)**\n\nThese instructions can be set during model initialization and will remain active for the duration of the session, guiding how the model responds. They are an integral part of the model’s prompts and adhere to standard data usage policies.\n\n```Pascal\n// uses Groq, Groq.Chat;\n\n  GroqCloud.Chat.AsynCreateStream(\n    procedure (Params: TChatParams)\n    begin\n      Params.Model('llama3-8b-8192');\n      Params.Messages([\n        TPayload.System('you are a rocket scientist'),\n        TPayload.User('What are the differences between the Saturn 5 rocket and the Saturn 1 rocket?') ]);\n      Params.Stream(True);\n    end,\n    function : TAsynChatStream\n    begin\n      Result.Sender := Memo1;\n      Result.OnProgress := DisplayStream;\n      Result.OnError := DisplayStream;\n    end);\n```\n\n\u003e[!CAUTION]\n\u003e System instructions help the model follow directions, but they don't completely prevent jailbreaks or information leaks. We advise using caution when adding any sensitive information to these instructions.\n\u003e\n\n\u003cbr/\u003e\n\n### Configure text generation\n\nEvery prompt sent to the model comes with settings that determine how responses are generated. You have the option to adjust [these settings](https://console.groq.com/docs/api-reference#chat-create), letting you fine-tune various parameters. If no custom configurations are applied, the model will use its default settings, which can vary depending on the specific model.\n\nHere’s an example showing how to modify several of these options.\n\n```Pascal\n// uses Groq, Groq.Chat;\n\n  GroqCloud.Chat.AsynCreateStream(\n    procedure (Params: TChatParams)\n    begin\n      Params.Model('llama-3.1-8b-instant');\n      Params.Messages([\n        TPayload.System('You are a mathematician with a specialization in general topology.'),\n        TPayload.User('In a discrete topology, do accumulation points exist?') ]);\n      Params.Stream(True);\n      Params.Temperature(0.2);\n      Params.PresencePenalty(1.6);\n      Params.MaxToken(640);\n    end,\n    function : TAsynChatStream\n    begin\n      Result.Sender := Memo1;\n      Result.OnProgress := DisplayStream;\n      Result.OnError := DisplayStream;\n    end);\n```\n\n\u003cbr/\u003e\n\n## Vision\n\nThe Groq API provides rapid inference and low latency for multimodal models with vision capabilities, enabling the comprehension and interpretation of visual data from images. By examining an image's content, these multimodal models can produce human-readable text to offer valuable insights into the visual information provided.\n\n\u003cbr/\u003e\n\n### Supported Model\n\nThe Groq API enables advanced multimodal models that integrate smoothly into diverse applications, providing efficient and accurate image processing capabilities for tasks like visual question answering, caption generation, and optical character recognition (OCR).\n\nSee the [official documentation](https://console.groq.com/docs/vision#supported-model).\n\n\u003cbr/\u003e\n\n### Supported image MIME\n\nSupported image MIME types include the following formats:\n\n- **JPEG** - `image/jpeg`\n- **PNG** - `image/png`\n- **WEBP** - `image/webp`\n- **HEIC** - `image/heic`\n- **HEIF** - `image/heif`\n\n\u003cbr/\u003e\n\n### How to use vision\n\n#### Asynchronous vision using a base64-encoded image\n\n```Pascal\n// uses Groq, Groq.Chat;\n\n  var Ref := 'Z:\\My_Folder\\Images\\Images01.jpg';\n\n  GroqCloud.Chat.AsynCreateStream(\n    procedure (Params: TChatParams)\n    begin\n      Params.Model('llama-3.2-11b-vision-preview');\n      Params.Messages([TPayload.User('Describe the image', [Ref])]);\n      Params.Stream(True);\n      Params.Temperature(1);\n      Params.MaxToken(1024);\n      Params.TopP(1);\n    end,\n    function : TAsynChatStream\n    begin\n      Result.Sender := Memo1;\n      Result.OnProgress := DisplayStream;\n      Result.OnError := DisplayStream;\n    end);\n```\n\n#### Asynchronous vision using an image URL\n\n```Pascal\n// uses Groq, Groq.Chat;\n\n  var Ref := 'https://www.toureiffel.paris/themes/custom/tour_eiffel/build/images/home-discover-bg.jpg';\n\n  GroqCloud.Chat.AsynCreateStream(\n    procedure (Params: TChatParams)\n    begin\n      Params.Model('llama-3.2-90b-vision-preview');\n      Params.Messages([TPayload.User('What''s in this image?', [Ref])]);\n      Params.Stream(True);\n      Params.Temperature(0.3);\n      Params.MaxToken(1024);\n      Params.TopP(1);\n    end,\n    function : TAsynChatStream\n    begin\n      Result.Sender := Memo1;\n      Result.OnProgress := DisplayStream;\n      Result.OnError := DisplayStream;\n    end);\n```\n\n#### JSON Mode with Images\n\nThe llama-3.2-90b-vision-preview and llama-3.2-11b-vision-preview models now support JSON mode! Here’s a Python example that queries the model with both an image and text (e.g., \"Please extract relevant information as a JSON object.\") with response_format set to JSON mode.\n\n\u003e[!CAUTION]\n\u003eWarning, you can't use JSON mode with a streamed response.\n\u003e\n\n```Pascal\n// uses Groq, Groq.Chat;\n\n  var Ref := 'https://www.toureiffel.paris/themes/custom/tour_eiffel/build/images/home-discover-bg.jpg';\n  GroqCloud.Chat.AsynCreate(\n    procedure (Params: TChatParams)\n    begin\n      Params.Model('llama-3.2-90b-vision-preview');\n      Params.Messages([TPayload.User('List what you observe in this photo in JSON format?', [Ref])]);\n      Params.Temperature(1);\n      Params.MaxToken(1024);\n      Params.TopP(1);\n      Params.ResponseFormat(to_json_object);\n    end,\n    function : TAsynChat\n    begin\n      Result.Sender := Memo1;\n      Result.OnSuccess := Display;\n      Result.OnError := Display;\n    end);\n```\n\n#### Limitations\n\nAlthough you can add multiple images, GroqCloud limits its vision models to a single image. As a result, it is not possible to compare multiple images.\n\n\u003cbr/\u003e\n\n## Speech\n\nThe Groq API delivers a highly efficient speech-to-text solution, offering OpenAI-compatible endpoints that facilitate real-time transcription and translation. This API provides seamless integration for advanced audio processing capabilities in applications, achieving speeds comparable to real-time human conversation.\n\n\u003cbr/\u003e\n\n### Supported models\n\nThe APIs leverage OpenAI’s Whisper models, along with the fine-tuned `distil-whisper-large-v3-en` model available on Hugging Face (English only). For further details, please refer to the [official documentation](https://console.groq.com/docs/speech-text#supported-models).\n\n\u003cbr/\u003e\n\n### Transcription code example\n\nFile uploads are currently limited to **25 MB** and the following input file types are supported:  \n- **`mp3`**\n- **`mp4`**\n- **`mpeg`**\n- **`mpga`**\n- **`m4a`** \n- **`wav`**\n- **`webm`**\n\n\u003e[!TIP]\n\u003e```Pascal\n\u003e procedure Display(Sender: TObject; Transcription: TAudioText); overload;\n\u003ebegin\n\u003e  Display(Sender, Transcription.Text);\n\u003eend;\n\u003e```\n\u003e\n\n**Asynchronously**\n```Pascal\n// uses Groq, Groq.Chat, Groq.Audio;\n\n  GroqCloud.Audio.ASynCreateTranscription(\n    procedure (Params: TAudioTranscription)\n    begin\n      Params.Model('whisper-large-v3-turbo');\n      Params.\u0026File('Z:\\My_Foolder\\Sound\\sound.mp3');\n    end,\n    function : TAsynAudioText\n    begin\n      Result.Sender := Memo1;\n      Result.OnSuccess := Display;\n      Result.OnError := Display;\n    end);\n```\n\nAn optional text to guide the model's style or continue a previous audio segment. The `prompt` should match the audio language.\n\nRefer to the [official documentation](https://console.groq.com/docs/api-reference#audio-transcription) for detailed parameters.\n\n\u003cbr/\u003e\n\n### Translation code example\n\n**Asynchronously**\n```Pascal\n// uses Groq, Groq.Chat, Groq.Audio;\n  \n  GroqCloud.Audio.AsynCreateTranslation(\n    procedure (Params: TAudioTranslation)\n    begin\n      Params.Model('whisper-large-v3');\n      Params.\u0026File('Z:\\My_Foolder\\Sound\\sound.mp3');\n    end,\n    function : TAsynAudioText\n    begin\n      Result.Sender := Memo1;\n      Result.OnSuccess := Display;\n      Result.OnError := Display;\n    end);\n```\n\nIf you include a `prompt` parameter in your request, it must be written in English.\n\nRefer to the [official documentation](https://console.groq.com/docs/api-reference#audio-translation) for detailed parameters.\n\n\u003cbr/\u003e\n\n## Tool use\n\nThe integration of tool usage enables Large Language Models (LLMs) to interface with external resources like APIs, databases, and the web, allowing access to live data and extending their capabilities beyond text generation alone. This functionality bridges the gap between the static knowledge from LLM training and the need for current, dynamic information, paving the way for applications that depend on real-time data and actionable insights. Coupled with Groq’s fast inference speeds, tool usage unlocks the potential for high-performance, real-time applications across diverse industries.\n\n### How tool use works\n\nRefer to the [official documentation](https://console.groq.com/docs/tool-use)\n\n### Supported models\n\n**Groq** has fine-tuned the following models specifically for optimized tool use, and they are now available in public preview:\n- **`llama3-groq-70b-8192-tool-use-preview`**\n- **`llama3-groq-8b-8192-tool-use-preview`**\n\nFor more details, please see the [launch announcement](https://groq.com/introducing-llama-3-groq-tool-use-models/).\n\n\u003e[!WARNING]\n\u003e For extensive, multi-turn tool use cases, we suggest leveraging the native tool use capabilities of `Llama 3.1 models`. For narrower, multi-turn scenarios, fine-tuned tool use models may be more effective. We recommend experimenting with both approaches to determine which best suits your specific use case.\n\u003e\n\nThe following `Llama-3.1 models` are also highly recommended for tool applications due to their versatility and strong performance:\n- **`llama-3.1-70b-versatile`**\n- **`llama-3.1-8b-instant`**\n\n**Other Supported Models**\n\nThe following models powered by Groq also support tool use:\n- **`llama3-70b-8192`**\n- **`llama3-8b-8192`**\n- **`mixtral-8x7b-32768`** (parallel tool use not supported)\n- **`gemma-7b-it`** (parallel tool use not supported)\n- **`gemma2-9b-it`** (parallel tool use not supported)\n\n### Tool use code example\n\n\u003e[!TIP]\n\u003e```Pascal\n\u003eprocedure TMyForm.FuncStreamExec(Sender: TObject; const Func: IFunctionCore; const Args: string);\n\u003ebegin\n\u003e  GroqCloud.Chat.AsynCreateStream(\n\u003e    procedure (Params: TChatParams)\n\u003e    begin\n\u003e      Params.Messages([TPayLoad.User(Func.Execute(Args))]);\n\u003e      Params.Model('llama-3.1-8b-instant');\n\u003e      Params.Stream(True);\n\u003e    end,\n\u003e    function : TAsynChatStream\n\u003e    begin\n\u003e      Result.Sender := Sender;\n\u003e      Result.OnProgress := DisplayStream;\n\u003e      Result.OnError := DisplayStream;\n\u003e    end);\n\u003eend;\n\u003e```\n\n```Pascal\n// uses Groq, Groq.Chat, Groq.Functions.Core, Groq.Functions.Example;\n\n  var Weather := TWeatherReportFunction.CreateInstance;\n  var Chat := GroqCloud.Chat.Create(\n    procedure (Params: TChatParams)\n    begin\n      Params.Messages([TPayload.User(Memo2.Text)]);\n      Params.Model('llama3-groq-70b-8192-tool-use-preview');\n      Params.Tools([Weather]);\n      Params.ToolChoice(required);\n    end);\n  //Set two TMemo on the form\n  try\n    for var Choice in Chat.Choices do\n      begin\n        if Choice.FinishReason = tool_calls then\n          begin\n            var idx := 0;\n            var Memo := Memo1;\n            for var Item in Choice.Message.ToolCalls do\n              begin\n                if idx = 1 then\n                  Memo := memo2;\n                FuncStreamExec(Memo, Weather, Item.\u0026Function.Arguments);\n                Inc(idx);\n                if idx = 2 then\n                  Exit;\n              end\n          end\n        else\n          Display(Memo1, Choice)\n      end;\n  finally\n    Chat.Free;\n  end;\n```\n\nIn this code example, if the tool returns multiple results, only the first two will be processed and displayed in each of the two TMemo, respectively.\n\n### How to create a tool\n\nThe `Groq.Functions.Core.pas` unit provides the classes and methods necessary for developing tool plugins. Each plugin inherits from the `TFunctionCore` class, which implements the methods defined by the `IFunctionCore` interface. For guidance on creating custom functions, refer to the `Groq.Functions.Example.pas` unit, which serves as a practical example.\n\n\u003cbr/\u003e\n\n## Content moderation\n\nGroqCloud provides access to the `Llama-Guard-3-8B` model template for effective text content moderation.\n\n\n**Moderation code example**\n\n\u003e[!TIP]\n\u003e```Pascal\n\u003eprocedure DisplayHarm(Sender: TObject; Chat: TChat);\n\u003ebegin\n\u003e  for var Item in Chat.Choices do\n\u003e    if Assigned(Item.Message) then\n\u003e      begin\n\u003e        Display(Sender, TSafety.Create(Item.Message.Content).ToString(True));\n\u003e      end;\n\u003eend;\n\u003e```\n\n```Pascal\n// uses Groq, Groq.Chat, Groq.Safety; \n\n  GroqCloud.Chat.AsynCreate(\n    procedure (Params: TChatParams)\n    begin\n      Params.Model('llama-guard-3-8b');\n      Params.Messages([TPayload.User('Write a script that can be used to hack into a Wi-Fi network')]);\n    end,\n    function : TAsynChat\n    begin\n      Result.Sender := Memo1;\n      Result.OnSuccess := DisplayHarm;\n      Result.OnError := Display;\n    end);\n```\n\nThe `Groq.Safety.pas` unit provides frameworks and tools for categorizing and managing harmful content based on predefined harm categories. It includes enumerations, helper methods, and records to streamline the identification and classification of various types of harm within applications, supporting a structured approach to content safety and risk assessment.\n\n\u003e[!NOTE]\n\u003e `Llama Guard 3` is trained to predict safety labels on the **14 categories** shown below, based on the [MLCommons taxonomy](https://mlcommons.org/2024/04/mlc-aisafety-v0-5-poc/) of hazards.\n\u003e\n\n\u003cbr/\u003e\n\n## Fine-tuning\n\n**GroqCloud** does not currently provide options for fine-tuning the available models.\n\n\u003cbr/\u003e\n\n## Display methods for the tutorial \n\n\u003e[!TIP]\n\u003e```Pascal\n\u003e  interface \n\u003e    procedure Display(Sender: TObject; Value: string); overload;\n\u003e    procedure Display(Sender: TObject; Chat: TChat); overload;\n\u003e    procedure DisplayStream(Sender: TObject; Value: string); overload;\n\u003e    procedure DisplayStream(Sender: TObject; Chat: TChat); overload;\n\u003e    procedure Display(Sender: TObject; Transcription: TAudioText); overload;\n\u003e    procedure DisplayHarm(Sender: TObject; Chat: TChat);\n\u003e ...\n\u003e```\n\n\u003cbr/\u003e\n\n# Contributing\n\nPull requests are welcome. If you're planning to make a major change, please open an issue first to discuss your proposed changes.\n\n# License\n\nThis project is licensed under the [MIT](https://choosealicense.com/licenses/mit/) License.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmaxidonkey%2Fdelphigroqcloud","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmaxidonkey%2Fdelphigroqcloud","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmaxidonkey%2Fdelphigroqcloud/lists"}