{"id":18162661,"url":"https://github.com/tinybiggames/lumina","last_synced_at":"2025-08-19T08:33:56.182Z","repository":{"id":260732646,"uuid":"882195186","full_name":"tinyBigGAMES/Lumina","owner":"tinyBigGAMES","description":"Local Generative AI","archived":false,"fork":false,"pushed_at":"2025-01-15T05:44:16.000Z","size":17505,"stargazers_count":34,"open_issues_count":2,"forks_count":5,"subscribers_count":9,"default_branch":"main","last_synced_at":"2025-08-17T07:36:02.817Z","etag":null,"topics":["gen-ai","gguf","llama-cpp","llm-inference","local-ai","pascal","win64","windows-10","windows-11"],"latest_commit_sha":null,"homepage":"","language":"Pascal","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tinyBigGAMES.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null},"funding":{"github":"tinyBigGAMES","patreon":null,"open_collective":null,"ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"lfx_crowdfunding":null,"polar":null,"buy_me_a_coffee":null,"thanks_dev":null,"custom":null}},"created_at":"2024-11-02T06:10:52.000Z","updated_at":"2025-06-09T18:36:10.000Z","dependencies_parsed_at":null,"dependency_job_id":"444fa968-36a8-44a1-b44e-43ec5c7d29ce","html_url":"https://github.com/tinyBigGAMES/Lumina","commit_stats":null,"previous_names":["tinybiggames/lumina"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/tinyBigGAMES/Lumina","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinyBigGAMES%2FLumina","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinyBigGAMES%2FLumina/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinyBigGAMES%2FLumina/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinyBigGAMES%2FLumina/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tinyBigGAMES","download_url":"https://codeload.github.com/tinyBigGAMES/Lumina/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinyBigGAMES%2FLumina/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":271121789,"owners_count":24702874,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-19T02:00:09.176Z","response_time":63,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["gen-ai","gguf","llama-cpp","llm-inference","local-ai","pascal","win64","windows-10","windows-11"],"created_at":"2024-11-02T10:04:51.643Z","updated_at":"2025-08-19T08:33:56.172Z","avatar_url":"https://github.com/tinyBigGAMES.png","language":"Pascal","funding_links":["https://github.com/sponsors/tinyBigGAMES"],"categories":[],"sub_categories":[],"readme":"![Lumina](media/lumina.png)  \n[![Chat on Discord](https://img.shields.io/discord/754884471324672040?style=for-the-badge)](https://discord.gg/tPWjMwK)\n[![Follow on Bluesky](https://img.shields.io/badge/Bluesky-tinyBigGAMES-blue?style=for-the-badge\u0026logo=bluesky)](https://bsky.app/profile/tinybiggames.com)\n\n# 🌟 Lumina: Advanced Local Generative AI for Delphi Developers 💻🤖\n\nLumina offers a cutting-edge 🛠️ for Delphi developers to seamlessly integrate advanced generative AI capabilities into their 📱. Built on the computational backbone of **llama.cpp** 🐪, Lumina prioritizes data privacy 🔒, performance ⚡, and a user-friendly API 📚, making it a powerful tool for local AI inference 🤖.\n\n## 🧐 Why Choose Lumina?\n\n- **Localized Processing** 🏠: Operates entirely offline, ensuring sensitive data remains confidential 🛡️ while offering complete computational control 🧠.\n- **Broad Model Compatibility** 🌐: Supports **GGUF models** compliant with llama.cpp standards, granting access to diverse AI architectures 🧩.\n- **Intuitive Development Interface** 🎛️: A concise, flexible API simplifies model management 🗂️, inference execution 🧮, and callback customization 🎚️, minimizing implementation complexity.\n- **Future-Ready Scalability** 🚀: This release emphasizes stability 🏗️ and foundational features, with plans for multi-turn conversation 💬 and retrieval-augmented generation (RAG) 🔍 in future updates.\n\n## 🛠️ Key Functionalities\n\n### 🤖 Advanced AI Integration\n\nLumina expands your development toolkit 🎒 with capabilities such as:\n- Dynamic chatbot creation 💬.\n- Automated text generation 📝 and summarization 📰.\n- Context-sensitive content generation ✍️.\n- Real-time inference for adaptive processes ⚡.\n\n### 🔒 Privacy-Driven AI Execution\n\n- Operates independently of external networks 🛡️, guaranteeing data security.\n- Uses Vulkan 🖥️ for optional GPU acceleration to enhance performance.\n\n### ⚙️ Performance Optimization\n\n- Configurable GPU utilization through the `AGPULayers` parameter 🧩.\n- Dynamic thread allocation based on hardware capabilities 🖥️ via `AMaxThreads`.\n- Comprehensive performance metrics 📊, offering insights into throughput 📈 and efficiency.\n\n### 🔗 Streamlined Integration\n\n- Embedded dependencies eliminate the need for external libraries 📦.\n- Lightweight architecture (~2.5MB overhead) ensures broad deployment compatibility 🌍.\n\n## 📥 Installation\n\n1. **Download the Repository** 📦\n   - [Download here](https://github.com/tinyBigGAMES/Lumina/archive/refs/heads/main.zip) and extract the files to your preferred directory 📂.\n\n2. **Acquire a GGUF Model** 🧠\n   - Obtain a model from [Hugging Face](https://huggingface.co), such as [Gemma 2.2B GGUF (Q8_0)](https://huggingface.co/bartowski/gemma-2-2b-it-abliterated-GGUF/resolve/main/gemma-2-2b-it-abliterated-Q8_0.gguf?download=true). Save it to a directory accessible to your application (e.g., `C:/LLM/GGUF`) 💾.\n\n3. **Ensure GPU Compatibility** 🎮\n   - Verify Vulkan compatibility for enhanced performance ⚡. Adjust `AGPULayers` as needed to accommodate VRAM limitations 📉.\n\n4. **✨ TLumina Class** \n   - 📜 Add `Lumina` to your `uses` section.  \n   - 🛠️ Create an instance of `TLumina`.  \n   - 🚀 All functionality will then be at your disposal. That simple! 🎉\n\n5. **Explore Examples** 🔍\n   - Check the `examples` directory for detailed usage demonstrations 📚.\n\n## 🛠️ Usage\n\n### 🔧 Basic Setup\n\nIntegrate Lumina into your Delphi project 🖥️:\n\n```delphi\nvar\n  Lumina: TLumina;\nbegin\n  Lumina := TLumina.Create;\n  try\n    if Lumina.LoadModel('C:\\LLM\\GGUF\\gemma-2-2b-it-abliterated-Q8_0.gguf',\n      '', 8192, -1, 8) then\n    begin\n      if Lumina.SimpleInference('What is the capital of Italy?') then\n        WriteLn('Inference completed successfully.')\n      else\n        WriteLn('Error: ', Lumina.GetError);\n    end;\n  finally\n    Lumina.Free;\n  end;\nend;\n```\n\n### 🎚️ Customizing Callbacks\n\nDefine custom behavior using Lumina’s callback functions 🛠️:\n\n```delphi\nprocedure NextTokenCallback(const AToken: string; const AUserData: Pointer);\nbegin\n  Write(AToken);\nend;\n\nLumina.SetNextTokenCallback(NextTokenCallback, nil);\n```\n\n## 📖 API Reference\n\n### 🧩 Core Methods\n\n- **LoadModel** 📂\n  - Parameters:\n    - `AModelFilename`: Path to the GGUF model file 📄.\n    - `ATemplate`: Optional inference template 📝.\n    - `AMaxContext`: Maximum context size (default: 512) 🧠.\n    - `AGPULayers`: GPU layer configuration (-1 for maximum) 🎮.\n    - `AMaxThreads`: Number of CPU threads allocated 🖥️.\n  - Returns a boolean indicating success ✅.\n\n- **SimpleInference** 🧠\n  - Accepts a single query for immediate processing 📝.\n  - Returns a boolean indicating success ✅.\n\n- **SetNextTokenCallback** 💬\n  - Assigns a handler to process tokens during inference 🧩.\n\n- **UnloadModel** ❌\n  - Frees resources allocated during model loading 🗑️.\n\n- **GetPerformanceResult** 📊\n  - Provides metrics, including token generation rates 📈.\n\n## 🛠️ Advanced Configurations\n\n### 🧠 Custom Inference Templates\n\nLumina will use the template defined in the model's meta data by default, but you can also define custom templates to match your model’s requirements or change its behavor. These are some common model templates ✍️:\n\n```delphi\nconst\n  CHATML_TEMPLATE = '\u003c|im_start|\u003e{role} {content}\u003c|im_end|\u003e\u003c|im_start|\u003eassistant';\n  GEMMA_TEMPLATE  = '\u003cstart_of_turn\u003e{role} {content}\u003cend_of_turn\u003e';\n  PHI_TEMPLATE    = '\u003c|{role}|\u003e {content}\u003c|end|\u003e\u003c|assistant|\u003e';\n```\n\n- **{role}** - will be replaced with the role (user, assistant, etc.)\n- **{content}** - will be replaced with the content sent to the model\n\n### 🎮 GPU Optimization\n\n- `AGPULayers` values:\n  - `-1`: Utilize all available layers (default) 🖥️.\n  - `0`: CPU-only processing 🖥️.\n  - Custom values for partial GPU utilization 🎛️.\n\n### 📊 Performance Metrics\n\nRetrieve detailed operational metrics 📈:\n\n```delphi\nvar\n  Perf: TLumina.PerformanceResult;\nbegin\n  Perf := Lumina.GetPerformanceResult;\n  WriteLn('Tokens/Sec: ', Perf.TokensPerSecond);\n  WriteLn('Input Tokens: ', Perf.TotalInputTokens);\n  WriteLn('Output Tokens: ', Perf.TotalOutputTokens);\nend;\n```\n\n## 🎙️ Media  \n\n### 🌊 Deep Dive Podcast  \nDiscover in-depth discussions and insights about **Lumina** and its innovative features. 🚀✨\n\nhttps://github.com/user-attachments/assets/165e3dee-b29f-4478-b9ef-4fb6d2df2485\n\n\n### 🛠️ Support and Resources\n\n- Report issues via the [Issue Tracker](https://github.com/tinyBigGAMES/Lumina/issues) 🐞.\n- Engage in discussions on the [Forum](https://github.com/tinyBigGAMES/Lumina/discussions) and [Discord](https://discord.gg/tPWjMwK) 💬.\n- Learn more at [Learn Delphi](https://learndelphi.org) 📚.\n\n### 🤝 Contributing  \n\nContributions to **✨ Lumina** are highly encouraged! 🌟  \n- 🐛 **Report Issues:** Submit issues if you encounter bugs or need help.  \n- 💡 **Suggest Features:** Share your ideas to make **Lumina** even better.  \n- 🔧 **Create Pull Requests:** Help expand the capabilities and robustness of the library.  \n\nYour contributions make a difference! 🙌✨\n\n#### Contributors 👥🤝\n\u003cbr/\u003e\n\n\u003ca href=\"https://github.com/tinyBigGAMES/Lumina/graphs/contributors\"\u003e\n  \u003cimg src=\"https://contrib.rocks/image?repo=tinyBigGAMES/Lumina\u0026max=500\u0026columns=20\u0026anon=1\" /\u003e\n\u003c/a\u003e\n\n### 📜 Licensing\n\n**Lumina** is distributed under the 🆓 **BSD-3-Clause License**, allowing for redistribution and use in both source and binary forms, with or without modification, under specific conditions. See the [LICENSE](https://github.com/tinyBigGAMES/Lumina?tab=BSD-3-Clause-1-ov-file#BSD-3-Clause-1-ov-file) file for more details.\n\n---\n\nAdvance your Delphi applications with Lumina 🌟 – a sophisticated solution for integrating local generative AI 🤖.\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"media/delphi.png\" alt=\"Delphi\"\u003e\n\u003c/p\u003e\n\u003ch5 align=\"center\"\u003e\n\nMade with :heart: in Delphi\n\u003c/h5\u003e\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftinybiggames%2Flumina","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftinybiggames%2Flumina","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftinybiggames%2Flumina/lists"}