This repo contains the code of the paper A universal 2-staten-action adaptive management solver

REQUIREMENTS:

This code is implemented in R and calls external packages. Users need to make sure that their version of R is 4.0.2 and the packages MDPtoolbox and data.table are installed.

Our package uses SARSOP (https://github.com/AdaCompNUS/sarsop ), which also needs to be priorly installed.

HOW TO?

• Build a hmMDP using MC-UAMS:

  • go to the folder "src/building hmMDP".
  • Open the "main.R" file.
  • Select the example to run (uncomment the corresponding data file)
  • Run "main.R"
  • The file runs MC-UAMS, and writes the ".pomdpx" file the folder "res/"
  • The ".pomdpx" can be used as input for MO-SARSOP (to be called outside R pomdpsol xxx); the resulting output file ".policyx" can be stored in the folder "data"

• Call the PUBD algorithm:

  • go to the folder "src/Dirichlet solver".
  • Open the "main.R" file.
  • Select the example to run (uncomment the corresponding data file)
  • Run "main.R"
  • The file runs PUBD and returns the optimal solution of the discrete MDP of hyperstates

• Run simulations:

  • make sure that the hmMDP solution (calculated using SARSOP)"xxx.policyx" file is in the folder "data/POLICYX"
  • go to the folder "src/simulations".
  • Open the "main.R" file.
  • Select the example to run (uncomment the corresponding data file)
  • Run "main.R"
  • The results of simulations are stored as csv files in the folder "res/"

• Analyse the results of simulations:

  • go to the folder "src/analyse results".
  • Open the "main.R" file.
  • Select the example that you wish to run (uncomment the corresponding data file)
  • Run "main.R"
  • The analysis of results is stored as a csv file in the folder "res/performance"

DESCRIPTION of THE FOLDERS

data contains the input data used in our paper:

• examples2SXA.R / potoroo.R /gouldian.R define the Adaptive Management problems (random and conservation – see supplementary material of details).
• gouldian4Exp.pomdpx is the original hmMDP assessed by 4 experts
• POLICYX:
  • XXX.policyx are the solution files returned by SARSOP.

res contains the outputs of simulations:

• expert_model:
  • For the case study of Gouldian finch, the files simXXX_expert_model.csv are the output of performance trajectories when the real model is included in the Experts set
• meanparams:
  • meanparamsXXX.csv are the parameters generated by MC-UAMS that are used to build our 2-state n-actions hmMDP (adaptive management solvers)
• POMDPX:
  • XXX.pomdpx are the POMDPX files obtained with MC-UAMS
• randomMDP:
  • XXX_randomMDP.csv describe the values of the true random MDPs that are used to assess MC-UAMS and PUBD
• simhmMDP
  • simhmMDPXXX.csv are the output of performance trajectories when applying MC-UAMS
• simopt
  • simoptXXX.csv are the output of performance trajectories when applying the true (unknown) optimal MDP policies
• simparams
  • simparamsXXX.csv are the output of performance trajectories when applying PUBD (see supplementary material for details about the algorithm).

src contains the source files:

• analyse results:
  • main.R : file to run to analyse the results
• building hmMDP:
  • generate_pomdpx.R : R wrapper to write the POMDPX file of a hmMDP.
  • main.R : file to run to build the POMDPX file of the examples in data (edit to select the problem)
  • mean_parameters.R : function to run Monte Carlo simulations that estimate the parameters of a MC-UAMS
• Dirichlet solver:
  • dirichlet_solver.R: main function to run the PUBD algorithm
  • main.R: main file to run to get the output of the PUBD algorithm
• simulations:
  • build_matrices_hmMDP.R : function to build the observation and reward matrix of a hmMDP
  • generate_random_mdp.R: function to draw uniformly MDPs
  • read_policyx.R : function to open policyx files
  • main.R : main file to run simulations of management
  • sim_mdp_momdp_policy.R : function to run simulations of management using the output of MC-UAMS
  • sim_mdp_parameter_uncertainty.R : function to run simulations of management using the output of PUBD (dirichlet_solver)