{"id":15361406,"url":"https://github.com/muhd-umer/rl-wireless","last_synced_at":"2025-04-15T07:31:04.683Z","repository":{"id":154877981,"uuid":"626045306","full_name":"muhd-umer/rl-wireless","owner":"muhd-umer","description":"RL-Wireless: Reinforcement learning-based resource allocation in wireless networks","archived":false,"fork":false,"pushed_at":"2024-10-09T11:32:50.000Z","size":11022,"stargazers_count":13,"open_issues_count":0,"forks_count":2,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-10-17T09:28:20.008Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/muhd-umer.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-04-10T17:21:32.000Z","updated_at":"2024-10-09T11:47:29.000Z","dependencies_parsed_at":null,"dependency_job_id":"d3b4c6d1-b37e-4fe0-92d6-c5c1d562228b","html_url":"https://github.com/muhd-umer/rl-wireless","commit_stats":{"total_commits":70,"total_committers":2,"mean_commits":35.0,"dds":0.3857142857142857,"last_synced_commit":"468f77b9f0b7d4afe7b37615e21aaa556f6f0184"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/muhd-umer%2Frl-wireless","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/muhd-umer%2Frl-wireless/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/muhd-umer%2Frl-wireless/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/muhd-umer%2Frl-wireless/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/muhd-umer","download_url":"https://codeload.github.com/muhd-umer/rl-wireless/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249026708,"owners_count":21200494,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-01T12:54:55.971Z","updated_at":"2025-04-15T07:31:01.382Z","avatar_url":"https://github.com/muhd-umer.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://raw.githubusercontent.com/muhd-umer/rl-wireless/refs/heads/main/resources/logo.png\" width=\"675\"\u003e\n\u003c/p\u003e\n\nThis repository contains code for implementing reinforcement learning-based resource allocation algorithms for wireless networks. The code is designed to optimize resource allocation in a dynamic wireless environment with changing network conditions and user demands.\n\nThe application of deep reinforcement learning (DRL) for dynamic resource allocation in wireless communication systems is explored in this project. An environment simulates a multi-cell massive MIMO wireless system. DRL algorithms such as DQN and PPO are used to optimize resource allocation, demonstrating improved efficiency over traditional methods. For more details, refer to the [report](./report/report.pdf).\n\n## Installation\n\nTo install the necessary dependencies and set up the project, follow these steps:\n\n### Clone the repository\n\n```sh\ngit clone https://github.com/muhd-umer/rl-wireless.git\ncd rl-wireless\n```\n\n### Create a new virtual environment\n\nIt is recommended to create a new virtual environment to avoid conflicts with other projects. You can create a new virtual environment using `conda` or `mamba`:\n\n**→ Using Conda**\n\n```sh\nconda create -n rl-wireless python=3.9\nconda activate rl-wireless\n\npip install -r requirements.txt\n```\n\n**→ Using Mamba**\n\n```sh\nwget -O miniforge.sh \\\n     \"https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-$(uname)-$(uname -m).sh\"\nbash miniforge.sh -b -p \"${HOME}/conda\"\n\nsource \"${HOME}/conda/etc/profile.d/conda.sh\"\nsource \"${HOME}/conda/etc/profile.d/mamba.sh\"\n\nconda activate\nmamba create -n rl-wireless python=3.9\nmamba activate rl-wireless\n\npip install -r requirements.txt\n```\n\n## Environment\n\nThe environment for this project is based on the `Gymnasium` library, a standard for creating RL environments. The main environment class is `MassiveMIMOEnv`, which is defined in the `network/environment.py` file.\n\n### Environment Setup\n\nTo register and create the environment, you can use the following code snippet:\n\n```python\nimport gymnasium as gym\nfrom network import MassiveMIMOEnv\nimport numpy as np\n\n# Set the parameters\nN = 7          # Number of cells (or base stations)\nM = 32         # Number of antennas per base station\nK = 10         # Number of user equipments (UEs) per cell\nNs = 10        # Number of samples for the channel realization\nmin_P = -20    # Minimum transmission power in dBm\nmax_P = 23     # Maximum transmission power in dBm\nnum_P = 10     # Number of discrete power levels\ndtype = np.float32    # Data type for computations\n\n# Register and create the environment\ngym.register(id=\"MassiveMIMO-v0\", entry_point=MassiveMIMOEnv)\n\nenv = gym.make(\n    \"MassiveMIMO-v0\",\n    N=N,\n    M=M,\n    K=K,\n    Ns=Ns,\n    min_P=min_P,\n    max_P=max_P,\n    num_P=num_P,\n    dtype=dtype,\n)\n```\n\n## Usage\n\nTo run the code and train an agent, you can use the provided notebook `run.ipynb`, which contains step-by-step instructions and examples. Below is an example of how to set up and run the environment:\n\n```python\nimport numpy as np\nimport gymnasium as gym\nfrom network import MassiveMIMOEnv\n\n# Create the environment\nenv = gym.make(\"MassiveMIMO-v0\", N=7, M=32, K=10, Ns=10, min_P=-20, max_P=23, num_P=10, dtype=np.float32)\n\n# Example usage\nstate = env.reset()\nfor _ in range(100):\n    action = env.action_space.sample()\n    next_state, reward, terminated, truncated, info = env.step(action)\n    if terminated or truncated:\n        state = env.reset()\n    else:\n        state = next_state\n```\n\n## Training\n\nThe `run.ipynb` notebook demonstrates how to train various DRL agents using the Ray RLlib library.\u003cbr\u003e\nIt trains PPO, DQN, and R2D2 agents and evaluates their performance.\n\n### Example: Training a PPO Agent\n\n```python\nimport ray\nfrom ray import air, tune\nfrom ray.tune.registry import get_trainable_cls\n\n# Initialize Ray\nray.init()\n\n# Configuration for PPO\nconfig = (\n    get_trainable_cls(\"PPO\")\n    .get_default_config()\n    .environment(\"MassiveMIMO-v0\")\n    .framework(\"torch\")\n    .resources(num_gpus=0.5)\n    .rollouts(num_rollout_workers=1)\n    .training(lr=tune.grid_search([0.005, 0.0001]))\n)\n\n# Training\nresults = tune.Tuner(\n    \"PPO\",\n    param_space=config.to_dict(),\n    run_config=air.RunConfig(stop={\"timesteps_total\": 100000}, local_dir=\"./results\"),\n).fit()\n```\n\n## Contributing\n\nContributions are always welcome and highly appreciated. 💖\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmuhd-umer%2Frl-wireless","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmuhd-umer%2Frl-wireless","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmuhd-umer%2Frl-wireless/lists"}