{"id":25373917,"url":"https://github.com/tinybiggames/aitoolkit","last_synced_at":"2025-04-09T09:15:46.074Z","repository":{"id":276914826,"uuid":"927923431","full_name":"tinyBigGAMES/AIToolkit","owner":"tinyBigGAMES","description":"AIToolkit - AI Construction Set","archived":false,"fork":false,"pushed_at":"2025-02-18T23:10:34.000Z","size":6995,"stargazers_count":13,"open_issues_count":0,"forks_count":2,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-04-05T10:35:25.523Z","etag":null,"topics":["delphi","genai","llamacpp","local-ai-development","win64"],"latest_commit_sha":null,"homepage":"","language":"Pascal","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tinyBigGAMES.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"tinyBigGAMES","patreon":null,"open_collective":null,"ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"lfx_crowdfunding":null,"polar":null,"buy_me_a_coffee":null,"thanks_dev":null,"custom":null}},"created_at":"2025-02-05T19:06:51.000Z","updated_at":"2025-03-09T21:37:32.000Z","dependencies_parsed_at":"2025-02-11T06:22:37.870Z","dependency_job_id":"acb50011-91a4-481b-afea-1a5b7823b063","html_url":"https://github.com/tinyBigGAMES/AIToolkit","commit_stats":null,"previous_names":["tinybiggames/aitoolkit"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinyBigGAMES%2FAIToolkit","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinyBigGAMES%2FAIToolkit/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinyBigGAMES%2FAIToolkit/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinyBigGAMES%2FAIToolkit/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tinyBigGAMES","download_url":"https://codeload.github.com/tinyBigGAMES/AIToolkit/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248008625,"owners_count":21032556,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["delphi","genai","llamacpp","local-ai-development","win64"],"created_at":"2025-02-15T03:19:49.070Z","updated_at":"2025-04-09T09:15:46.068Z","avatar_url":"https://github.com/tinyBigGAMES.png","language":"Pascal","readme":"![AIToolkit](media/aitoolkit.png)  \n[![Chat on Discord](https://img.shields.io/discord/754884471324672040?style=for-the-badge)](https://discord.gg/tPWjMwK)\n[![Follow on Bluesky](https://img.shields.io/badge/Bluesky-tinyBigGAMES-blue?style=for-the-badge\u0026logo=bluesky)](https://bsky.app/profile/tinybiggames.com)  \n\n## 🚀 AIToolkit – Unlock Local Generative AI in Delphi!\n\n🔹 **AIToolkit** is a powerful yet intuitive framework for **Delphi** developers, enabling **local generative AI** directly on your desktop—no cloud, no dependencies, just pure **on-device AI**!  \n\n⚡ **Powered by llama.cpp**, AIToolkit provides a **lightweight, efficient**, and **extendable** set of **classes and routines**, giving you full control over how you integrate AI into your applications. Whether you're building **chatbots 🤖, creative content generators ✨, or AI-powered tools 🔧**, this toolkit serves as your **foundation** to innovate without limits.  \n\n💡 **Easy to use, simple yet elegant**, AIToolkit lets you customize and expand its features **to your heart’s desire**—tailor it to fit your needs and build something truly unique!  \n\n🔍 Here's a sneak peek at some of AIToolkit’s powerful features: Pre-built classes for messages, inference, function calling (tools), utilities, and more—giving you a solid foundation to build and expand upon! 🚀\n\n🔥 **Start creating with AIToolkit today and bring AI to your Delphi projects!**  \n\n👉 **Ready to dive in?** [Get started now!](https://github.com/tinyBigGAMES/AIToolkit/archive/refs/heads/main.zip) 🚀 (*see the usage notes in `UTestbed.pas`*)\n\n---\n\n**🚧 Note:** This repository is currently in the setup phase, and documentation is not yet available. However, the code is **fully functional** and generally **stable**. Stay tuned—this README, along with the documentation and other resources, will be updated soon! 🚀\n\n---\n\n## 🛠️ Key Functionalities\n\n### 🖥️ Model Loading \u0026 GPU Optimization  \n- AIToolkit allows you to load **multiple models** across **CPU, GPU, or a hybrid setup**.  \n- Control GPU usage with:  \n  - 🎯 **`MainGPU`** – Select which GPU device to use.  \n  - 📊 **`GPULayers`** – Define how many layers to load on the GPU:  \n    - `GPULayers = 0` ➝ Model runs fully on **CPU**.  \n    - `GPULayers = N` ➝ N layers load onto **GPU**, the rest remain in **system memory**.  \n    - `MainGPU = -1 \u0026 GPULayers = -1` ➝ **Auto-selects the best GPU** and loads the model fully onto it.  \n\n#### 📥 Download \u0026 Setup  \n1. **Download the model** from our **Hugging Face** account:  \n   - [DeepSeek-R1-Distill-Llama-8B-Abliterated (Q4_K_M-GGUF)](https://huggingface.co/tinybiggames/DeepSeek-R1-Distill-Llama-8B-abliterated-Q4_K_M-GGUF/resolve/main/deepseek-r1-distill-llama-8b-abliterated-q4_k_m.gguf?download=true)  \n2. **Place the model in your desired directory** (example directory used in code):  \n   - 📂 `C:/LLM/GGUF`  \n\n### 🔄 Always Up-to-Date with Llama.cpp  \n- ✅ Stays aligned with the **latest llama.cpp** releases for **performance \u0026 compatibility improvements**.  \n\n### 🛠️ Modular Kit-Based Design  \n- Everything is in **kit form** (Messages, Inference, Tools, etc.), so you can **mix, match, and extend** as needed.  \n\n### 🔍 Web Search Integration  \n- 🌐 Powered by **Tavily.com** for **seamless web search capabilities**.  \n- 🆓 **Get your search API key** from [Tavily](https://tavily.com/)  \n  - 🎁 Includes **1,000 free tokens per month**  \n  - 🔑 **Set up your API key**:  \n    - Create an **environment variable** named **`TAVILY_API_KEY`** and assign it to your API key.  \n\n### ⚙️ Automated Tool Registration  \n- 🔧 **Auto-registers tools** from published **static class methods**.  \n- 📜 **Generates JSON schema** required for LLM tool execution—no manual setup needed!  \n\n## 📺 Media\n\n🌊 Deep Dive Podcast  \nDiscover in-depth discussions and insights about AIToolkit and its innovative features. 🚀✨\n\nhttps://github.com/user-attachments/assets/fc59e7ee-446f-43ad-889f-28b55dc6bb96\n\n🚀 AIToolkit in Action!  \n🌐 Showing how it will use the 🔎 **web_search** tool to get the latest 📢 up-to-date information.\n\nhttps://github.com/user-attachments/assets/14c306d2-114e-4121-8171-a5c500a804b8\n\n## 🛠️ Support and Resources\n\n- 🐞 **Report issues** via the [Issue Tracker](https://github.com/tinyBigGAMES/AIToolkit/issues).\n- 💬 **Engage in discussions** on the [Forum](https://github.com/tinyBigGAMES/AIToolkit/discussions) and [Discord](https://discord.gg/tPWjMwK).\n- 📚 **Learn more** at [Learn Delphi](https://learndelphi.org).\n\n## 🤝 Contributing  \n\nContributions to **✨ AIToolkit** are highly encouraged! 🌟  \n- 🐛 **Report Issues:** Submit issues if you encounter bugs or need help.  \n- 💡 **Suggest Features:** Share your ideas to make **AIToolkit** even better.  \n- 🔧 **Create Pull Requests:** Help expand the capabilities and robustness of the library.  \n\nYour contributions make a difference! 🙌✨\n\n#### Contributors 👥🤝\n\u003cbr/\u003e\n\n\u003ca href=\"https://github.com/tinyBigGAMES/AIToolkit/graphs/contributors\"\u003e\n  \u003cimg src=\"https://contrib.rocks/image?repo=tinyBigGAMES/AIToolkit\u0026max=250\u0026columns=10\u0026anon=1\" /\u003e\n\u003c/a\u003e\n\n## 📜 Licensing\n\n**AIToolkit** is distributed under the **🆓 BSD-3-Clause License**, allowing for redistribution and use in both source and binary forms, with or without modification, under specific conditions.  \nSee the [📜 LICENSE](https://github.com/tinyBigGAMES/AIToolkit?tab=BSD-3-Clause-1-ov-file#BSD-3-Clause-1-ov-file) file for more details.\n\n---\n\n🛠️ AIToolkit – Empower Your Delphi Projects with Local AI, Simple, Fast, and Limitless! 🚀🤖\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"media/delphi.png\" alt=\"Delphi\"\u003e\n\u003c/p\u003e\n\u003ch5 align=\"center\"\u003e\n  \nMade with ❤️ in Delphi  \n","funding_links":["https://github.com/sponsors/tinyBigGAMES"],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftinybiggames%2Faitoolkit","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftinybiggames%2Faitoolkit","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftinybiggames%2Faitoolkit/lists"}