{"id":13457503,"url":"https://github.com/semitable/lb-foraging","last_synced_at":"2025-09-20T14:05:44.648Z","repository":{"id":34587891,"uuid":"139773449","full_name":"semitable/lb-foraging","owner":"semitable","description":"Level-based Foraging (LBF): A multi-agent environment for RL","archived":false,"fork":false,"pushed_at":"2024-09-15T17:06:11.000Z","size":381,"stargazers_count":188,"open_issues_count":14,"forks_count":68,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-08-30T11:19:23.134Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/semitable.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-07-04T23:27:39.000Z","updated_at":"2025-07-31T13:57:04.000Z","dependencies_parsed_at":"2024-09-15T18:27:38.342Z","dependency_job_id":"b4263a45-ef3c-4f91-b2ab-169364d498f2","html_url":"https://github.com/semitable/lb-foraging","commit_stats":{"total_commits":134,"total_committers":7,"mean_commits":"19.142857142857142","dds":"0.30597014925373134","last_synced_commit":"a409b672bf1fab3ba1999156ccfce5ccdb58443b"},"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/semitable/lb-foraging","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/semitable%2Flb-foraging","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/semitable%2Flb-foraging/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/semitable%2Flb-foraging/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/semitable%2Flb-foraging/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/semitable","download_url":"https://codeload.github.com/semitable/lb-foraging/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/semitable%2Flb-foraging/sbom","scorecard":{"id":810835,"data":{"date":"2025-08-11","repo":{"name":"github.com/semitable/lb-foraging","commit":"79f383b7db0a3530f14252d5b6cc0d85aa9fb385"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":3.6,"checks":[{"name":"Code-Review","score":2,"reason":"Found 2/7 approved changesets -- score normalized to 2","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Warn: no topLevel permission defined: .github/workflows/python-publish.yml:1","Info: no jobLevel write permissions found"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/python-publish.yml:16: update your workflow using https://app.stepsecurity.io/secureworkflow/semitable/lb-foraging/python-publish.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/python-publish.yml:18: update your workflow using https://app.stepsecurity.io/secureworkflow/semitable/lb-foraging/python-publish.yml/master?enable=pin","Warn: pipCommand not pinned by hash: .github/workflows/python-publish.yml:23","Warn: pipCommand not pinned by hash: .github/workflows/python-publish.yml:24","Info:   0 out of   2 GitHub-owned GitHubAction dependencies pinned","Info:   0 out of   2 pipCommand dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: MIT License: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 26 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-23T13:08:26.177Z","repository_id":34587891,"created_at":"2025-08-23T13:08:26.177Z","updated_at":"2025-08-23T13:08:26.177Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":276106042,"owners_count":25586191,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-20T02:00:10.207Z","response_time":63,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T09:00:27.598Z","updated_at":"2025-09-20T14:05:44.617Z","avatar_url":"https://github.com/semitable.png","language":"Python","funding_links":[],"categories":["Other reading material (blogs, websites, videos)","Papers"],"sub_categories":["Opponent Modelling","ICLR 2022"],"readme":"\n\u003cp align=\"center\"\u003e\n \u003cimg width=\"350px\" src=\"docs/img/logo.png\" align=\"center\" alt=\"Level Based Foraging (LBF)\" /\u003e\n \u003cp align=\"center\"\u003eA multi-agent reinforcement learning environment\u003c/p\u003e\n\u003c/p\u003e\n\n\u003c!-- TABLE OF CONTENTS --\u003e\n\u003ch1\u003e Table of Contents \u003c/h1\u003e\n\n- [About The Project](#about-the-project)\n- [Getting Started](#getting-started)\n  - [Installation](#installation)\n- [Usage](#usage)\n  - [Observation Space](#observation-space)\n  - [Action space](#action-space)\n  - [Rewards](#rewards)\n- [Human Play](#human-play)\n- [Please Cite](#please-cite)\n- [Contributing](#contributing)\n- [Contact](#contact)\n\n\n\u003e [!CAUTION]\n\u003e The LBF environment was updated to support the new [Gymnasium](https://gymnasium.farama.org/) interface in replacement of the deprecated `gym=0.21` dependency (many thanks @LukasSchaefer). For backwards compatibility, please see [Gymnasium compatibility documentation](https://gymnasium.farama.org/content/gym_compatibility/) or use version v1.1.1 of the repository. The main changes to the interface are as follows:\n\u003e - `obss = env.reset()` --\u003e `obss, info = env.reset()`\n\u003e - `obss, rewards, dones, info = env.step(actions)` --\u003e `obss, rewards, done, truncated, info = env.step(actions)`\n\u003e - The `done` flag is now given as a single boolean value instead of a list of booleans.\n\u003e - You can give the reset function a particular seed with `obss, info = env.reset(seed=42)` to initialise a particular episode.\n\n\n\u003c!-- ABOUT THE PROJECT --\u003e\n# About The Project\n\nThis environment is a mixed cooperative-competitive game, which focuses on the coordination of the agents involved. Agents navigate a grid world and collect food by cooperating with other agents if needed.\n\n\u003cp align=\"center\"\u003e\n \u003cimg width=\"450px\" src=\"docs/img/lbf.gif\" align=\"center\" alt=\"Level Based Foraging (LBF) illustration\" /\u003e\n\u003c/p\u003e\n\nMore specifically, agents are placed in the grid world, and each is assigned a level. Food is also randomly scattered, each having a level on its own. Agents can navigate the environment and can attempt to collect food placed next to them. The collection of food is successful only if the sum of the levels of the agents involved in loading is equal to or higher than the level of the food. Finally, agents are awarded points equal to the level of the food they helped collect, divided by their contribution (their level). The figures below show two states of the game, one that requires cooperation, and one more competitive.\n\n\nWhile it may appear simple, this is a very challenging environment, requiring the cooperation of multiple agents while being competitive at the same time. In addition, the discount factor also necessitates speed for the maximisation of rewards. Each agent is only awarded points if it participates in the collection of food, and it has to balance between collecting low-levelled food on his own or cooperating in acquiring higher rewards. In situations with three or more agents, highly strategic decisions can be required, involving agents needing to choose with whom to cooperate. Another significant difficulty for RL algorithms is the sparsity of rewards, which causes slower learning.\n\nThis is a Python simulator for level based foraging. It is based on OpenAI's RL framework, with modifications for the multi-agent domain. The efficient implementation allows for thousands of simulation steps per second on a single thread, while the rendering capabilities allows humans to visualise agent actions. Our implementation can support different grid sizes or agent/food count. Also, game variants are implemented, such as cooperative mode (agents always need to cooperate) and shared reward (all agents always get the same reward), which is attractive as a credit assignment problem.\n\n\n\n\u003c!-- GETTING STARTED --\u003e\n# Getting Started\n\n## Installation\n\nInstall using pip\n```sh\npip install lbforaging\n```\nOr to ensure that you have the latest version:\n```sh\ngit clone https://github.com/semitable/lb-foraging.git\ncd lb-foraging\npip install -e .\n```\n\n\n\u003c!-- USAGE EXAMPLES --\u003e\n# Usage\n\nCreate environments with the gym framework.\nFirst import\n```python\nimport lbforaging\n```\n\nThen create an environment:\n```python\nenv = gym.make(\"Foraging-8x8-2p-1f-v3\")\n```\n\nWe offer a variety of environments using this template:\n```\n\"Foraging-{GRID_SIZE}x{GRID_SIZE}-{PLAYER COUNT}p-{FOOD LOCATIONS}f{-coop IF COOPERATIVE MODE}-v0\"\n```\n\nBut you can register your own variation using (change parameters as needed):\n```python\nfrom gym.envs.registration register\n\nregister(\n    id=\"Foraging-{0}x{0}-{1}p-{2}f{3}-v3\".format(s, p, f, \"-coop\" if c else \"\"),\n    entry_point=\"lbforaging.foraging:ForagingEnv\",\n    kwargs={\n        \"players\": p,\n        \"max_player_level\": 3,\n        \"field_size\": (s, s),\n        \"max_food\": f,\n        \"sight\": s,\n        \"max_episode_steps\": 50,\n        \"force_coop\": c,\n    },\n)\n```\n\nSimilarly to Gym, but adapted to multi-agent settings step() function is defined as\n```python\nnobs, nreward, ndone, ninfo = env.step(actions)\n```\n\nWhere n-obs, n-rewards, n-done and n-info are LISTS of N items (where N is the number of agents). The i'th element of each list should be assigned to the i'th agent.\n\n\n\n## Observation Space\n\n## Action space\n\nactions is a LIST of N INTEGERS (one of each agent) that should be executed in that step. The integers should correspond to the Enum below:\n\n```python\nclass Action(Enum):\n    NONE = 0\n    NORTH = 1\n    SOUTH = 2\n    WEST = 3\n    EAST = 4\n    LOAD = 5\n```\nValid actions can always be sampled like in a gym environment, using:\n```python\nenv.action_space.sample() # [2, 3, 0, 1]\n```\nAlso, ALL actions are valid. If an agent cannot move to a location or load, his action will be replaced with `NONE` automatically.\n\n## Rewards\n\nThe rewards are calculated as follows. When one or more agents load a food, the food level is rewarded to the agents weighted with the level of each agent. Then the reward is normalised so that at the end, the sum of the rewards (if all foods have been picked-up) is one. \nIf you prefer code:\n\n```python\nfor a in adj_players: # the players that participated in loading the food\n    a.reward = float(a.level * food) # higher-leveled agents contribute more and are rewarded more. \n    if self._normalize_reward:\n        a.reward = a.reward / float(\n            adj_player_level * self._food_spawned\n        )  # normalize reward so that the final sum of rewards is one.\n```\n\n\n\u003c!-- HUMAN PLAY SCRIPT --\u003e\n# Human Play\n\nWe also provide a simple script that allows you to play the environment as a human. This is useful for debugging and understanding the environment dynamics. To play the environment, run the following command:\n```sh\npython human_play.py --env \u003cenv_name\u003e\n```\nwhere `\u003cenv_name\u003e` is the name of the environment you want to play. For example, to play an LBF task with two agents and one food in a 8x8 grid, run:\n```sh\npython human_play.py --env Foraging-8x8-2p-1f-v3\n```\n\nWithin the script, you can control a single agent at the time using the following keys:\n- Arrow keys: move current agent up/ down/ left/ right\n- L: load food\n- K: load food and let agent keep loading (even if agent is swapped)\n- SPACE: do nothing\n- TAB: change the current agent (rotates through all agents)\n- R: reset the environment and start a new episode\n- H: show help\n- D: display agent info (at every time step)\n- ESC: exit\n\n\n\u003c!-- CITATION --\u003e\n# Please Cite\n1. The paper that first uses this implementation of Level-based Foraging (LBF) and achieves state-of-the-art results:\n```\n@inproceedings{christianos2020shared,\n  title={Shared Experience Actor-Critic for Multi-Agent Reinforcement Learning},\n  author={Christianos, Filippos and Schäfer, Lukas and Albrecht, Stefano V},\n  booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},\n  year={2020}\n}\n```\n2. A comperative evaluation of cooperative MARL algorithms and includes an introduction to this environment:\n```\n@inproceedings{papoudakis2021benchmarking,\n   title={Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Tasks},\n   author={Georgios Papoudakis and Filippos Christianos and Lukas Schäfer and Stefano V. Albrecht},\n   booktitle = {Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks (NeurIPS)},\n   year={2021},\n   openreview = {https://openreview.net/forum?id=cIrPX-Sn5n},\n}\n```\n\n\u003c!-- CONTRIBUTING --\u003e\n# Contributing\n\n1. Fork the Project\n2. Create your Feature Branch (`git checkout -b feature/AmazingFeature`)\n3. Commit your Changes (`git commit -m 'Add some AmazingFeature`)\n4. Push to the Branch (`git push origin feature/AmazingFeature`)\n5. Open a Pull Request\n\n\n\u003c!-- CONTACT --\u003e\n# Contact\n\nFilippos Christianos - f.christianos@ed.ac.uk\n\nProject Link: [https://github.com/semitable/lb-foraging](https://github.com/semitable/lb-foraging)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsemitable%2Flb-foraging","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsemitable%2Flb-foraging","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsemitable%2Flb-foraging/lists"}