{"id":18497808,"url":"https://github.com/containers/podman-desktop-extension-ai-lab","last_synced_at":"2025-05-16T14:05:33.422Z","repository":{"id":217273992,"uuid":"733663692","full_name":"containers/podman-desktop-extension-ai-lab","owner":"containers","description":"Work with LLMs on a local environment using containers","archived":false,"fork":false,"pushed_at":"2025-05-13T14:26:03.000Z","size":13781,"stargazers_count":223,"open_issues_count":223,"forks_count":59,"subscribers_count":21,"default_branch":"main","last_synced_at":"2025-05-13T15:47:54.191Z","etag":null,"topics":["ai","containers","inference-server","llms","local","podman"],"latest_commit_sha":null,"homepage":"https://podman-desktop.io/extensions/ai-lab","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/containers.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"CODE-OF-CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":"CODEOWNERS","security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-12-19T20:59:01.000Z","updated_at":"2025-05-13T14:26:08.000Z","dependencies_parsed_at":"2024-01-22T11:35:35.783Z","dependency_job_id":"3283d583-ae13-4fdc-a6f4-989ce5b855fa","html_url":"https://github.com/containers/podman-desktop-extension-ai-lab","commit_stats":{"total_commits":1266,"total_committers":28,"mean_commits":"45.214285714285715","dds":0.5631911532385466,"last_synced_commit":"75598c6d970901c7821151b38987d347ea758de3"},"previous_names":["projectatomic/studio-extension"],"tags_count":21,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/containers%2Fpodman-desktop-extension-ai-lab","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/containers%2Fpodman-desktop-extension-ai-lab/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/containers%2Fpodman-desktop-extension-ai-lab/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/containers%2Fpodman-desktop-extension-ai-lab/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/containers","download_url":"https://codeload.github.com/containers/podman-desktop-extension-ai-lab/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254544146,"owners_count":22088807,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","containers","inference-server","llms","local","podman"],"created_at":"2024-11-06T13:35:58.917Z","updated_at":"2025-05-16T14:05:33.376Z","avatar_url":"https://github.com/containers.png","language":"TypeScript","funding_links":[],"categories":["*Ops for AI","Table of content","Model Serving \u0026 Inference"],"sub_categories":["Model Serving \u0026 Inference","Generative AI","Model Serving Frameworks"],"readme":"# Podman AI Lab\n\nPodman AI Lab is an open source extension for Podman Desktop to work with LLMs (Large Language Models) on a local environment. Featuring a recipe catalog with common AI use cases, a curated set of open source models, and a playground for learning, prototyping and experimentation, Podman AI Lab helps you to quickly and easily get started bringing AI into your applications, without depending on infrastructure beyond your laptop ensuring data privacy and security.\n\n\n## Topics\n- [Technology](#technology)\n- [Extension features](#extension-features)\n- [Requirements](#requirements)\n- [Installation](#installation)\n- [Usage](#usage)\n- [Contributing](#contributing)\n- [Feedback](#feedback)\n\n\n\n## Technology\n\nPodman AI Lab uses [Podman](https://podman.io) machines to run inference servers for LLM models and AI applications.\nThe AI models can be downloaded, and common formats like [GGUF](https://github.com/ggerganov/ggml/blob/master/docs/gguf.md), [Pytorch](https://pytorch.org) or [Tensorflow](https://www.tensorflow.org) are supported.\n\n## Extension features\n\n### AI Models\n\nPodman AI Lab provides a curated list of open source AI models and LLMs. Once downloaded, the models are available to be used for AI applications, model services and playgrounds.\n\n#### Model services\n\nOnce a model is downloaded, a model service can be started. A model service is an inference server that is running in a container and exposing the model through the well-known chat API common to many providers.\n\n#### Playgrounds\n\nThe integrated Playground environments allow for experimenting with available models in a local environment. An intuitive user prompt helps in exploring the capabilities and accuracy of various models and aids in finding the best model for the use case at hand. The Playground interface further allows for parameterizing models to further optimize the settings and attributes of each model.\n\n### AI applications\n\nOnce an AI model is available through a well known endpoint, it's easy to imagine a new world of applications that will connect and use the AI model. Podman AI Lab supports AI applications as a set of containers that are connected together.\n\nPodman AI Lab ships with a so-called Recipes Catalog that helps you navigate a number of core AI use cases and problem domains such as Chat Bots, Code Generators and Text Summarizers. Each recipe comes with detailed explanations and sample applications that can be run with various large language models (LLMs). Experimenting with multiple models allows finding the optimal one for your use case.\n\n## Requirements\n\n### Requirement 1. Software and hardware requirements\n\n**OS:**\n\nCompatible on Windows, macOS \u0026 Linux\n\n**Software:**\n\n- [Podman Desktop 1.8.0+](https://github.com/containers/podman-desktop)\n- [Podman 4.9.0+](https://github.com/containers/podman)\n\n**Hardware**\n\nLLMs AI models are heavy resource consumers both in terms of memory and CPU. Each of the provided models consumes about 4GiB of memory and requires at least 4 CPUs to run.\n\nSo we recommend that a minimum of 12GB of memory and at least 4 CPUs for the Podman machine.\n\nAs an additional recommended practice, do nor run more than 3 simultaneous models concurrently.\n\nPlease note that this is not relevant for WSL on Windows as the WSL technology the memory and CPU with the host desktop.\n\n## Installation\n\nYou can install the Podman AI Lab extension directly inside of Podman Desktop.\n\nGo to Extensions \u003e Catalog \u003e Install Podman AI Lab.\n\n![](https://github.com/containers/podman-desktop-media/raw/ai-lab/gifs/install_ai_lab.gif)\n\nTo install a development version, use the `Install custom...` action as shown in the recording below.\n\nThe name of the image to use is `ghcr.io/containers/podman-desktop-extension-ai-lab`. You can get released tags for the image at https://github.com/containers/podman-desktop-extension-ai-lab/pkgs/container/podman-desktop-extension-ai-lab.\n\n![](https://github.com/containers/podman-desktop-media/raw/ai-lab/gifs/install_development_version.gif)\n\n## Usage\n\n1. **Download a model**\n\nLet's select a model from the catalog and download it locally to our workstation.\n\n![](https://github.com/containers/podman-desktop-media/raw/ai-lab/gifs/download-model.gif)\n\n2. **Start an inference server**\n\nOnce a model is available locally, let's start an inference server\n\n![](https://github.com/containers/podman-desktop-media/raw/ai-lab/gifs/start-inference-server.gif)\n\n3. **Start a playground to have a chat conversation with model**\n\n![](https://github.com/containers/podman-desktop-media/raw/ai-lab/gifs/playground.gif)\n\n4. **Start an AI application and use it from the browser**\n\n![](https://github.com/containers/podman-desktop-media/raw/ai-lab/gifs/start-ai-app.gif)\n\n## Contributing\n\nWant to help develop and contribute to Podman AI Lab?\n\nYou can use `pnpm watch --extension-folder` from the Podman Desktop directory to automatically rebuild and test the AI Lab extension:\n\n\u003e **_Note_**: make sure you have the appropriate [pre-requisits](https://github.com/containers/podman-desktop/blob/main/CONTRIBUTING.md#prerequisites-prepare-your-environment)\n  installed.\n\n```sh\ngit clone https://github.com/containers/podman-desktop\ngit clone https://github.com/containers/podman-desktop-extension-ai-lab\ncd podman-desktop-extension-ai-lab\ncorepack enable pnpm\npnpm install\npnpm build\ncd ../podman-desktop\npnpm watch --extension-folder ../podman-desktop-extension-ai-lab/packages/backend\n```\n\nIf you are live editing the frontend package, from packages/frontend folder:\n\n```\n$ pnpm watch\n```\n\n### Cleaning up resources\n\nWe'll be adding a way to let a user cleanup their environment: see issue https://github.com/containers/podman-desktop-extension-ai-lab/issues/469.\nFor the time being, please consider the following actions:\n1. Remove the extension from Podman Desktop, from the Settings \u003e Extensions\n2. Remove the running playground environments from the list of Pods\n3. Remove the images built by the recipes\n4. Remove the containers related to AI\n5. Cleanup your local clone of the recipes: `$HOME/podman-desktop/ai-lab`\n\n### 📖 Providing a custom catalog\n\nThe extension provides by default a curated list of recipes, models and categories. However, this system is extensible and you can define your own.\n\nTo enhance the existing catalog, you can create a file located in the extension storage folder `$HOME/.local/share/containers/podman-desktop/extensions-storage/redhat.ai-lab/user-catalog.json`.\n\nIt must follow the same format as the default catalog [in the sources of the extension](https://github.com/containers/podman-desktop-extension-ai-lab/blob/main/packages/backend/src/assets/ai.json).\n\n\u003e :information_source: The default behaviour is to append the items of the user's catalog to the default one.\n\n\u003e :warning: Each item (recipes, models or categories) has a unique id, when conflict between the default catalog and the user one are found, the user's items overwrite the defaults.\n\n### Packaging sample applications\n\nSample applications may be added to the catalog. See [packaging guide](https://github.com/containers/podman-desktop-extension-ai-lab/blob/main/PACKAGING-GUIDE.md) for detailed information.\n\n## Roadmap\n\nThe roadmap is always open and we are looking for your feedback. Please create new issues and upvote on the issues that are feeling the most important for you.\n\nWe will be working on the following items:\n- **Expanded Recipes**: Discover new use cases and samples to inspire and accelerate your applications.\n- **GPU Acceleration**: Speeding up processing times by leveraging GPU acceleration.\n- **API/CLI**: Interact with Podman AI Lab from CLI and APIs.\n- **Enhanced Playgrounds**: Streamlined workflows and UX giving a better space to experiment with LLMs and quickly iterate.\n- **Fine Tuning with [InstructLab](https://instructlab.ai/)**: Re-train LLMs with a set of taxonomy knowledges. Learn more about [the InstructLab project](https://github.com/instructlab).\n- **Enable Function Calling**: Use LLMs to retrieve or interact with external tool by doing API calls.\n- **Local RAG**: Explore RAG pattern, load your document and test behavior of the model.\n- **Bridge with AI Platforms (incl. K8s)**: Connect to remote models and ease deployment of applications.\n\n## Feedback\n\nYou can provide your feedback on the extension with [this form](https://forms.gle/tctQ4RtZSiMyQr3R8) or create [an issue on this repository](https://github.com/containers/podman-desktop-extension-ai-lab/issues).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcontainers%2Fpodman-desktop-extension-ai-lab","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcontainers%2Fpodman-desktop-extension-ai-lab","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcontainers%2Fpodman-desktop-extension-ai-lab/lists"}