https://github.com/dylan-asmar/estimated_joint_belief
Code supporting the paper Efficient Multiagent Planning via Shared Action Suggestions.
https://github.com/dylan-asmar/estimated_joint_belief
collaboration dec-pomdp multiagent-planning
Last synced: 11 months ago
JSON representation
Code supporting the paper Efficient Multiagent Planning via Shared Action Suggestions.
- Host: GitHub
- URL: https://github.com/dylan-asmar/estimated_joint_belief
- Owner: dylan-asmar
- Created: 2024-10-09T18:38:17.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2024-12-31T18:01:03.000Z (over 1 year ago)
- Last Synced: 2025-05-12T11:51:45.929Z (about 1 year ago)
- Topics: collaboration, dec-pomdp, multiagent-planning
- Language: Julia
- Homepage:
- Size: 814 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: readme.md
Awesome Lists containing this project
README
# Supplementary Materials for Efficient Multiagent Planning via Shared Action Suggestions
This repository contains the code and supplementary materials for the paper "Efficient Multiagent Planning via Shared Action Suggestions".
[arXiv Paper](https://arxiv.org/abs/2412.11430)
## Repository Structure
```
├── src/
│ ├── problems.jl # Defines default parameters for the problems
│ ├── simulate.jl # Contains code for running individual simulations
│ ├── simulation_runner.jl # Runs all simulations
│ ├── policy_generator.jl # Used to generate additional policies
│ ├── output_data.jl # Displays results in a table
│ ├── policies/ # Contains a subset of generated policies
│ └── results/ # Contains simulation results
│
├── MultiAgentPOMDPProblems/
│ └── src/
│ ├── BoxPush/ # Cooperative Box Push
│ ├── JointMeet/ # Meeting in a Grid
│ ├── broadcast_channel.jl # Broadcast Channel
│ ├── multi_tiger_pomdp.jl # Dec-Tiger
│ ├── multi_wireless.jl # Wireless Network
│ ├── sotchastic_mars.jl # Mars Rover
│ ├── vis_utils.jl # Visualization Utilities
│ └── MultiAgentPOMDPProblems.jl # Package definition
|
├── install.jl # Installs the package
├── README.md # This file
├── Project.toml # Defines the package requirements
└── Appendix A.pdf # Additional results (average and maximum size of belief sets)
```
## Installation
All of the developement was done with Julia 1.10 and Julia 1.11. We recommend using the most up to date stable release, but it should work with Julia 1.8 or later. You can download Julia [here](https://julialang.org/downloads/).
To setup your environment, run the following from the root directory:
```julia
julia> include("install.jl")
```
or manually:
```julia
julia> using Pkg
julia> Pkg.activate(".")
julia> Pkg.instantiate()
julia> Pkg.develop(path="./MultiAgentPOMDPProblems")
julia> Pkg.precompile()
```
## Problem Implementation Details
For details on the implemenation of the problem used, reference the `MultiAgentPOMDPProblems` package. The parameters used for each problem can be found in the `problems.jl` file.
## Generating Policies
Only a subset of policies are included due to memory constraints. To generate additional policies, use `src/policy_generator.jl`. The parameters used to generate the policies are in `PROBLEMS_TO_RUN` with only the time limit being changed from the SARSOP parameters. Most policices converged prior to the time limit.
## Running Simulations
To run single simulations and visualize how the belief changes with the sharing of suggestions, use `src/simulate.jl`.
### Example Usage
```julia
using POMDPs
using POMDPTools
using MultiAgentPOMDPProblems
using ProgressMeter
using Printf
using JLD2
using DiscreteValueIteration # needed for loading the MMDP policy
# Used for handling results
using CSV
using Logging
include("src/problems.jl")
include("src/suggested_action_policies.jl")
include("src/simulate.jl")
include("src/problem_and_policy_helpers.jl")
problem_symbol = :tiger_3
control_option = :conflate_action
joint_problem, agent_problems, joint_policy, agent_policies, joint_mdp_policy = load_policy(problem_symbol)
joint_control = get_controller(:mpomdp, joint_problem, joint_policy, agent_problems, agent_policies) # Used to visualize the joint belief for comparison
# joint_control = nothing # This should be used when not wanting a visualization of the joint policy
control = get_controller(control_option, joint_problem, joint_policy, agent_problems, agent_policies; delta_single=1e-5, delta_joint=1e-5, max_beliefs=200)
seed = 42
s0 = rand(MersenneTwister(seed), initialstate(joint_problem))
results = run_simulation(
joint_problem, s0, control;
seed=seed,
text_output=false,
max_steps=10,
show_plots=true,
joint_control=joint_control
)
```
### Example Usage (Meeting in a 2 x 2 Grid UI, WP)
```julia
# Same packages as above and included files
problem_symbol = :joint_meet_2x2_ui_wp
control_option = :conflate_action
joint_problem, agent_problems, joint_policy, agent_policies, joint_mdp_policy = load_policy(problem_symbol)
joint_control = get_controller(:mpomdp, joint_problem, joint_policy, agent_problems, agent_policies) # Used to visualize the joint belief for comparison
# joint_control = nothing # This should be used when not wanting a visualization of the joint policy
control = get_controller(control_option, joint_problem, joint_policy, agent_problems, agent_policies; delta_single=1e-5, delta_joint=1e-5, max_beliefs=200)
seed = 43
s0 = rand(MersenneTwister(seed), initialstate(joint_problem))
results = run_simulation(
joint_problem, s0, control;
seed=seed,
text_output=false,
max_steps=10,
show_plots=true,
joint_control=joint_control
)
```
## Paper Experiments
To recreate all experiments in the paper, run `src/simulation_runner.jl`. This requires policies for all of the problems and must be computed using `src/policy_generator.jl`.
The csv for the experiments is located at `src/results/results_2024-10-15.csv`. To view the results in a table format, run `src/output_data.jl`.