{"id":25692683,"url":"https://github.com/tinybiggames/sophora","last_synced_at":"2025-07-29T11:33:43.994Z","repository":{"id":279177220,"uuid":"933045104","full_name":"tinyBigGAMES/Sophora","owner":"tinyBigGAMES","description":"Sophora - AI Reasoning, Function-calling \u0026 Knowledge Retrieval","archived":false,"fork":false,"pushed_at":"2025-02-28T05:13:49.000Z","size":7518,"stargazers_count":21,"open_issues_count":0,"forks_count":2,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-07-21T21:13:29.440Z","etag":null,"topics":["delphi","function-calling","knowledge-retrieval","llamacpp","local-genai","reasoning","win64"],"latest_commit_sha":null,"homepage":"","language":"Pascal","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tinyBigGAMES.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"tinyBigGAMES","patreon":null,"open_collective":null,"ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"lfx_crowdfunding":null,"polar":null,"buy_me_a_coffee":null,"thanks_dev":null,"custom":null}},"created_at":"2025-02-15T02:43:19.000Z","updated_at":"2025-04-28T23:36:37.000Z","dependencies_parsed_at":"2025-02-24T08:34:29.369Z","dependency_job_id":null,"html_url":"https://github.com/tinyBigGAMES/Sophora","commit_stats":null,"previous_names":["tinybiggames/sophora"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/tinyBigGAMES/Sophora","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinyBigGAMES%2FSophora","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinyBigGAMES%2FSophora/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinyBigGAMES%2FSophora/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinyBigGAMES%2FSophora/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tinyBigGAMES","download_url":"https://codeload.github.com/tinyBigGAMES/Sophora/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinyBigGAMES%2FSophora/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":267678448,"owners_count":24126333,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-29T02:00:12.549Z","response_time":2574,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["delphi","function-calling","knowledge-retrieval","llamacpp","local-genai","reasoning","win64"],"created_at":"2025-02-24T23:28:11.209Z","updated_at":"2025-07-29T11:33:43.988Z","avatar_url":"https://github.com/tinyBigGAMES.png","language":"Pascal","readme":"![Sophora](media/sophora.png)  \n[![Chat on Discord](https://img.shields.io/discord/754884471324672040?style=for-the-badge)](https://discord.gg/tPWjMwK)\n[![Follow on Bluesky](https://img.shields.io/badge/Bluesky-tinyBigGAMES-blue?style=for-the-badge\u0026logo=bluesky)](https://bsky.app/profile/tinybiggames.com)  \n\n**Sophora** is a local generative AI toolkit for **Delphi**, powered by the **DeepHermes-3** model and the latest **llama.cpp** optimizations. It enables fast, efficient, and unified reasoning, making it ideal for AI-driven applications that require **high-performance local inference** without relying on external cloud services. With features like **function calling**, **embedding generation**, **retrieval-augmented generation (RAG)**, and deep inference capabilities, Sophora provides developers with a versatile and powerful toolset for integrating AI into their Delphi projects. By supporting **optimized execution on modern hardware**, including **compute capability 5.0+ GPUs** via **Vulkan** for acceleration, it ensures smooth and efficient model operations.\n\n## 🚀 Key Features\n- **Local AI Inference**: Run **DeepHermes-3** (Llama 3-based) entirely on your machine, enabling fully offline AI capabilities.\n- **Fast Token Streaming**: Supports both **non-thinking** (fast response) and **thinking** (deep reasoning) modes.\n- **Function Calling \u0026 Embeddings**: Execute **function calls** and perform **vector-based search** for advanced AI-driven workflows.\n- **Retrieval-Augmented Generation (RAG)**: Enhances AI-generated responses using structured database lookups.\n- **SQL and Vector Databases**: Works with **SQLite3** and vector stores, making structured and semantic searches more efficient.\n- **Optimized with llama.cpp**: Leverages the latest optimizations for **high performance and reduced memory usage**.\n- **Flexible Model Deployment**: Supports various model configurations, letting users balance between performance and accuracy.\n\n## 📥 Getting Started\n\n### 1️⃣ Download and Install Sophora\nGet the latest version of Sophora and set up the toolkit:\n\n- Download the latest version from: [Sophora Main ZIP](https://github.com/tinyBigGAMES/Sophora/archive/refs/heads/main.zip) or clone the repository:\n  ```sh\n  git clone https://github.com/tinyBigGAMES/Sophora.git\n  ```\n- Extract the contents to your preferred directory.\n- Open the project in **Delphi**, and run the provided examples to explore the toolkit. Be sure to reference the **Usage Notes** in `UTestbed.pas` for insights about setup and using the toolkit.\n- Ensure your system meets the minimum requirements for running large language models efficiently. Your device will need enough RAM/VRAM to hold the model plus context. Your GPU must have compute capability 5.0+ and support Vulkan for acceleration.\n\n### 2️⃣ Download the Model\nSophora requires **DeepHermes-3**, which can be downloaded from **Hugging Face**:\n- [DeepHermes-3-Llama-3-8B-Preview-abliterated-Q4_K_M-GGUF](https://huggingface.co/tinybiggames/DeepHermes-3-Llama-3-8B-Preview-abliterated-Q4_K_M-GGUF/resolve/main/deephermes-3-llama-3-8b-preview-abliterated-q4_k_m.gguf?download=true) (General, Reasoning, Tools)\n- [bge-m3-Q8_0-GGUF](https://huggingface.co/tinybiggames/bge-m3-Q8_0-GGUF/resolve/main/bge-m3-q8_0.gguf?download=true) (Embeddings)\n- Place the downloaded model in the desired location (default: `C:/LLM/GGUF`).\n- Ensure the model file is correctly placed before running the inference engine.\n\n### 3️⃣ Setup Search API (Optional)\nTo enable web-augmented search capabilities, obtain an API key from [Tavily](https://tavily.com/).\n\n- You receive **1000 free API credits per month**.\n- Set an environment variable:\n  ```sh\n  TAVILY_API_KEY=\"your_api_key_here\"\n  ```\n- This API can be used for enhanced external queries via tool calls when needed.\n\n## 🛠️ Usage Examples\n\n### 🔹 Basic AI Query (Non-Thinking Mode)\nSophora can generate **fast responses** without deep reasoning.\n```delphi\nLMsg := TsoMessages.Create();\nLInf := TsoInference.Create();\nif not LInf.LoadModel() then Exit;\nLMsg.Add(soUser, 'Who is Bill Gates?');\nif not LInf.Run(LMsg) then\n  soConsole.PrintLn(LInf.GetError());\n```\n\n### 🔹 Deep Thinking Mode\nSophora enables **multi-step AI reasoning** for complex problem-solving.\n```delphi\nLMsg.Add(soSystem, 'You are a deep-thinking AI...');\nLMsg.Add(soUser, 'Solve this riddle: I walk on four legs in the morning...');\nLInf.Run(LMsg);\n```\n\n### 🔹 Embedding Generation\nSophora supports **vector search** using LLM embeddings.\n```delphi\nLEmb := TsoEmbeddings.Create();\nLEmb.LoadModel();\nLResult := LEmb.Generate('Explain data analysis in ML');\n```\n\n### 🔹 Retrieval-Augmented Generation (RAG)\nStore and retrieve **articles** from an SQLite database.\n```delphi\nLDb := TsoDatabase.Create();\nLDb.Open('articles.db');\nLDb.ExecuteSQL('INSERT INTO articles VALUES (''AI is transforming industries.'')');\nLDb.ExecuteSQL('SELECT * FROM articles');\n```\n\n### 🔹 Vector Database Search\nSophora supports **semantic search** over stored documents.\n```delphi\nLEmb := TsoEmbeddings.Create();\nLEmb.LoadModel();\nLVectorDB := TsoVectorDatabase.Create();\nLVectorDB.Open(LEmb, 'vectors.db');\nLVectorDB.AddDocument('doc1', 'AI and deep learning research.');\nLSearchResults := LVectorDB.Search('machine learning', 3);\n```\n\n## 📊 Performance Metrics\nSophora provides **detailed performance tracking**:\n- **Input Tokens**: Number of tokens processed.\n- **Output Tokens**: Tokens generated by the model.\n- **Speed**: Processing speed in tokens per second.\n\n### ✅ Example Performance Output:\n```plaintext\nPerformance:\nInput : 15 tokens\nOutput: 156 tokens\nSpeed : 49.68 tokens/sec\n```\n\n## ⚠️ Repository Status\n🚧 **Note:** This repository is currently in the **setup phase**, and full documentation is not yet available. However, the code is fully functional and generally stable. Additional **examples, guides, and API documentation** will be added soon. Stay tuned—this README, along with other resources, will be continuously updated! 🚀\n\n## 📺 Media\n🌊 Deep Dive Podcast  \nDiscover in-depth discussions and insights about Sophora and its innovative features. 🚀✨\n\n\nhttps://github.com/user-attachments/assets/6e82bf55-34fc-4085-8f97-0e0faca50a47\n\n\n\n## 🛠️ Support and Resources\n\n- 🐞 **Report issues** via the [Issue Tracker](https://github.com/tinyBigGAMES/Sophora/issues).\n- 💬 **Engage in discussions** on the [Forum](https://github.com/tinyBigGAMES/Sophora/discussions) and [Discord](https://discord.gg/tPWjMwK).\n- 📚 **Learn more** at [Learn Delphi](https://learndelphi.org).\n\n## 🤝 Contributing  \n\nContributions to **✨ Sophora** are highly encouraged! 🌟  \n- 🐛 **Report Issues:** Submit issues if you encounter bugs or need help.  \n- 💡 **Suggest Features:** Share your ideas to make **Sophora** even better.  \n- 🔧 **Create Pull Requests:** Help expand the capabilities and robustness of the library.  \n\nYour contributions make a difference! 🙌✨\n\n#### Contributors 👥🤝\n\u003cbr/\u003e\n\n\u003ca href=\"https://github.com/tinyBigGAMES/Sophora/graphs/contributors\"\u003e\n  \u003cimg src=\"https://contrib.rocks/image?repo=tinyBigGAMES/Sophora\u0026max=500\u0026columns=20\u0026anon=1\" /\u003e\n\u003c/a\u003e\n\n## 📜 Licensing\n\n**Sophora** is distributed under the **🆓 BSD-3-Clause License**, allowing for redistribution and use in both source and binary forms, with or without modification, under specific conditions.  \nSee the [📜 LICENSE](https://github.com/tinyBigGAMES/Sophora?tab=BSD-3-Clause-1-ov-file#BSD-3-Clause-1-ov-file) file for more details.\n\n## 💖 Sponsoring\n\nIf you find this project useful, please consider [sponsoring this project](https://github.com/sponsors/tinyBigGAMES). Your support helps sustain development, improve features, and keep the project thriving.\n\nIf you're unable to support financially, there are many other ways to contribute:\n- ⭐ **Star the repo** – It helps increase visibility and shows appreciation.\n- 📢 **Spread the word** – Share the project with others who might find it useful.\n- 🐛 **Report bugs** – Help improve the project by identifying and reporting issues.\n- 🔧 **Submit fixes** – Found a bug? Fix it and contribute!\n- 💡 **Make suggestions** – Share ideas for improvements and new features.\n\nEvery contribution, big or small, helps make this project better. Thank you for your support! 🚀\n\n\n---\n\n🛠️ Sophora AI Toolkit – A Powerful Local AI Framework for Delphi with Fast Token Streaming, Deep Reasoning, RAG, and Vector Search! 🚀🤖\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"media/delphi.png\" alt=\"Delphi\"\u003e\n\u003c/p\u003e\n\u003ch5 align=\"center\"\u003e\n  \nMade with ❤️ in Delphi  \n\n\n","funding_links":["https://github.com/sponsors/tinyBigGAMES"],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftinybiggames%2Fsophora","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftinybiggames%2Fsophora","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftinybiggames%2Fsophora/lists"}