{"id":18243817,"url":"https://github.com/achronus/velora","last_synced_at":"2025-11-07T00:03:19.132Z","repository":{"id":261093101,"uuid":"883206276","full_name":"Achronus/velora","owner":"Achronus","description":"A Liquid Reinforcement Learning (RL) framework","archived":false,"fork":false,"pushed_at":"2025-04-05T14:41:25.000Z","size":2848,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-05T15:25:23.924Z","etag":null,"topics":["gymnasium","liquid-neural-networks","model-free","mujoco","oop","python","pytorch","reinforcement-learning"],"latest_commit_sha":null,"homepage":"https://velora.achronus.dev/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Achronus.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-11-04T15:01:10.000Z","updated_at":"2025-04-01T16:29:49.000Z","dependencies_parsed_at":"2025-02-23T20:25:24.384Z","dependency_job_id":"66bc0a56-cd1a-4bab-8a88-c844ed922217","html_url":"https://github.com/Achronus/velora","commit_stats":{"total_commits":20,"total_committers":1,"mean_commits":20.0,"dds":0.0,"last_synced_commit":"9abab8f4f5c6c7fd6c30fb44210da966ba2b1532"},"previous_names":["achronus/velora"],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Achronus%2Fvelora","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Achronus%2Fvelora/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Achronus%2Fvelora/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Achronus%2Fvelora/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Achronus","download_url":"https://codeload.github.com/Achronus/velora/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247900841,"owners_count":21015138,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["gymnasium","liquid-neural-networks","model-free","mujoco","oop","python","pytorch","reinforcement-learning"],"created_at":"2024-11-05T09:03:22.304Z","updated_at":"2025-11-07T00:03:19.102Z","avatar_url":"https://github.com/Achronus.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"![Logo](https://raw.githubusercontent.com/Achronus/velora/main/docs/assets/imgs/main.jpg)\n\n[![codecov](https://codecov.io/gh/Achronus/velora/graph/badge.svg?token=OF7WP5Q9PT)](https://codecov.io/gh/Achronus/velora)\n![Python Version](https://img.shields.io/pypi/pyversions/velora)\n![License](https://img.shields.io/github/license/Achronus/velora)\n![Issues](https://img.shields.io/github/issues/Achronus/velora)\n\nFound on:\n\n- [PyPi](https://pypi.org/project/velora)\n- [GitHub](https://github.com/Achronus/velora)\n\n# Velora\n\n**Velora** is a lightweight and modular framework built on top of powerful libraries like [Gymnasium](https://gymnasium.farama.org/) and [PyTorch](https://pytorch.org/). It is home to a new type of RL agent called ***NeuroFlow*** (NF) that specializes in Autonomous Cyber Defence through a novel Deep Reinforcement Learning (RL) approach we call ***Liquid RL***.\n\n## Benefits\n\n- **Explainability**: NF agents use [Liquid Neural Networks](https://arxiv.org/abs/2006.04439) (LNNs) and [Neural Circuit Policies](https://arxiv.org/abs/1803.08554) (NCPs) to model Cyber system dynamics, not just data patterns. Also, they use sparse NCP connections to mimic biological efficiency, enabling clear, interpretable strategies via a labeled Strategy Library.\n- **Adaptability**: NF agents dynamically grow their networks using a fitness score, adding more neurons to a backbone only when new Cyber strategies emerge, keeping agents compact and robust.\n- **Planning**: NF agents use a Strategy Library and learned environment model to plan strategic sequences for proactive Cyber defense.\n- **Always Learning**: using [EWC](https://arxiv.org/abs/1612.00796), NF agents refine existing strategies and learn new ones post-training, adapting to evolving Cyber threats like new attack patterns.\n- **Customizable**: NF agents are PyTorch-based, designed to be intuitive, easy to use, and modular so you can easily build your own!\n\n## Installation\n\nTo get started, simply install it through [pip](https://pypi.org/project/velora) using one of the options below.\n\n### GPU Enabled\n\nFor [PyTorch](https://pytorch.org/get-started/locally/) with CUDA (recommended):\n\n```bash\npip install torch torchvision velora --extra-index-url https://download.pytorch.org/whl/cu126\n```\n\n### CPU Only\n\nOr, for [PyTorch](https://pytorch.org/get-started/locally/) with CPU only:\n\n```bash\npip install torch torchvision velora\n```\n\n## Example Usage\n\nHere's a simple example that should work 'as is':\n\n```python\nfrom velora.models import NeuroFlow, NeuroFlowCT\nfrom velora.utils import set_device\n\n# Setup PyTorch device\ndevice = set_device()\n\n# For continuous tasks\nmodel = NeuroFlowCT(\n    \"InvertedPendulum-v5\",\n    20,  # actor neurons \n    128,  # critic neurons\n    device=device,\n    seed=64,  # remove for automatic generation\n)\n\n# For discrete tasks\nmodel = NeuroFlow(\n    \"CartPole-v1\",\n    20,  # actor neurons \n    128,  # critic neurons\n    device=device,\n)\n\n# Train the model using a batch size of 64\nmodel.train(64, n_episodes=50, display_count=10)\n```\n\nCurrently, the framework only supports [Gymnasium](https://gymnasium.farama.org/) environments and is planned to expand to [PettingZoo](https://pettingzoo.farama.org/index.html) for Multi-agent (MARL) tasks, with updated adaptations of [CybORG](https://github.com/cage-challenge/CybORG/tree/main) environments.\n\n## API Structure\n\nThe frameworks API is designed to be simple and intuitive. We've broken into two main categories: `core` and `extras`.\n\n### Core\n\nThe primary building blocks you'll use regularly.\n\n```python\nfrom velora.models import [algorithm]\nfrom velora.callbacks import [callback]\n```\n\n### Extras\n\nUtility methods that you may use occasionally.\n\n```python\nfrom velora.gym import [method]\nfrom velora.utils import [method]\n```\n\n## Active Development\n\n🚧 View the [Roadmap](https://velora.achronus.dev/starting/roadmap) 🚧\n\n**Velora** is a tool that is continuously being developed. There's still a lot to do to make it a great framework, such as detailed API documentation, and expanding our NeuroFlow agents.\n\nOur goal is to provide a quality open-source product that works 'out-of-the-box' that everyone can experiment with, and then gradually fix unexpected bugs and introduce more features on the road to a `v1` release.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fachronus%2Fvelora","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fachronus%2Fvelora","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fachronus%2Fvelora/lists"}