Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/ealonsogzl/MuSA

MuSA: The Multiscale Snow data Assimilation System
https://github.com/ealonsogzl/MuSA

Last synced: 3 months ago
JSON representation

MuSA: The Multiscale Snow data Assimilation System

Awesome Lists containing this project

README 

        
# MuSA: The Multiple Snow data Assimilation System   

  

The Multiscale Snow Assimilation System (MuSA), is a flexible data  assimilation toolbox for experimental and operational snowpack  reanalysis development. MuSA was designed to fuse gridded observations  with an ensemble of simulations generated by the Flexible Snow Model  [(FSM2)](https://github.com/RichardEssery/FSM2) by using different  Bayesian based data assimilation algorithms. 



In its current version, it also offers support for the Snow17 model and a simple temperature index model. Potentially other numerical models could be implemented (not necessarily limited to snow models).



![alt text](https://github.com/ealonsogzl/MuSA/blob/master/img/PBS_animation.gif)

 Figure 1: Comparison between open loop and updated simulation after assimilating UAV snow depth retrievals at 5m resolution 

### Inputs  

  

The inputs of MuSA are composed by meteorological forcing and  observations to be assimilated. Both the forcing and observations must  share the **same geometry**, with the same resolution and number of  cells in the latitudinal and longitudinal axes, and should be provided in the [netCDF](https://www.unidata.ucar.edu/software/netcdf/) format. In this version, the meteorological forcing must be provided in an hourly timestep. Optionally it is  possible to provide a mask with the same geometry of the mandatory input  files to avoid to run MuSA over certain cells of your domain. The  meteorological forcing needed for running MuSA is composed by: 

- Incident shortwave radiation (W m-2)

- Incident longwave radiation ( W m-2)

- Precipitation  (Kg-2 m-2 s-1) 

- Temperature (K) 

- Relative Humidity (%) 

- Wind speed  (m s-1) 

- Atmospheric pressure (Pa) 

  

In its current version MuSA provides support for assimilating different  variables. Note that it is possible to provide more than one of the  following variables at the same time, i.e. MuSA has support for joint  assimilation experiments. In its current version, MuSA is able to assimilate: 

- SWE (mm) 

- Snow depth (m) 

- Land/Snow surface temperature (K) 

- Fractional snow cover area (-) 

- Albedo (-)

- Sensible heat flux to the atmosphere (W m-2)

- Latent heat flux to the atmosphere (W m-2)



The support of other variables like liquid water content, density, ice content etc.  could be relatively easily implemented on demand. 

  

### Data assimilation algorithms

There are different data assimilation and resampling algorithms implemented in MuSA.  Some testing should be done when developing data assimilation experiments, as the performance may be different depending on the problem to solve, and regarding the literature there is not a clear winner. Also, the computational cost will be different, and may be a strong conditioner in some situations.



Filters:

-   Particle Filter (PF)

-   Ensemble Kalman filter (EnKF)

-   Iterative ensemble Kalman filter (IEnKF)



Smoothers:

-   Particle batch smoother (PBS)

-   Ensemble smoother (ES)

-   Iterative ensemble smoother (IES)

-   Particle-adjusted iterative ensemble smoother (PIES)

-   Robust Adaptive Metropolis initialised by IES (IES-MCMC)

    

Resampling (for particle filters only):

-   Bootstrapping

-   Residual resampling

-   Stratified resampling

-   Systematic resampling

-   Redraw from a normal approximation of the posterior



### Outputs

The outputs of MuSA are pickles compressed with blosc (the latid_lonid.pkl.bl format). Each file stores a dictionary containing the following dataframes:

-  **OL_Sim**: Contains the reference simulation (snow simulation without any assimilation).

-  **updated_Sim**: Contains the updated simulation after the assimilation, i.e. weighted mean of the ensemble of simulations.

-  **sd_Sim**: Contains the weighted standard deviation of the ensemble after the assimilation.

-  **DA_Results**: Contains information about the observed variables and posterior parameters (in the normal space).

-  **prior_sd**: Contains the prior standar deviation of the ensemble (no assimilation)

-  **prior_mean**: Contains the prior mean of the ensemble (no assimilation)



You can open these files by your own means, or by using the following function:



```

from modules.internal_fns.py import  io_read



my_cell = io_read(./RESULTS/latid_lonid.pkl.bl)

```



**lonid** and **latid** are the longitude and latitude ids of each cell of the simulation.



Additionally it is possible to store the ensembles generated for each cell. This is an optional feature as it may be a bit memory consuming. However It may be useful in some circumstances specially for advanced users. The ensembles will be stored in the same format, and will containing [Ensemble objects](https://github.com/ealonsogzl/MuSA/blob/master/modules/internal_class.py).



### Usage



MuSA works on GNU/Linux (and therefore Mac) based platforms. MuSA has been tested also in Windows using the Windows Subsystem for Linux (WSL). MuSA relies on python3 with the usual scientific libraries (numpy, pandas, scipy...) and netCDF4 installed. You will also need to have gfortran in the path. The easiest way to do this is to generate a dedicated conda environment. You can use the [MuSAenv.yml](https://github.com/ealonsogzl/MuSA/blob/master/MuSAenv.yml) file of the repository to create the conda environment:



```

conda env create --name MuSAenv --file=MuSAenv.yml

```



Then for running MuSA simply:



```

conda activate MuSAenv

python main.py

```

Or if MPI is activated in the [config.py](https://github.com/RichardEssery/FSM2) file:

```

conda activate MuSAenv

mpiexec -n nprocess python -m mpi4py.futures main.py

```

Where nprocess is the number of threads to be deployed.



These commands should run the reproducible example included in the repository. This example contains all the information needed by MuSA. It is composed of a few cells containing meteorological forcing and drone SfM derived snowdepth information. To change the configuration of MuSA, you should modify the [config.py](https://github.com/ealonsogzl/MuSA/blob/master/config.py) file. Also it is possible to modify the way MuSA generates the ensemble by modifying the [constants.py](https://github.com/ealonsogzl/MuSA/blob/master/constants.py) file.

An [example script](https://github.com/ealonsogzl/MuSA/blob/master/run_PBS.sh) is also provided to run MuSA in distributed supercomputing facilities using PBS (Portable Batch System, not Particle Batch Smoother :wink:) or [Slurm](https://github.com/ealonsogzl/MuSA/blob/master/run_slurm.sh) arrays. The use of PBS/Slurm arrays is compatible with both MPI and multiprocessing. The use of PBS/Slurm arrays is compatible with both MPI and multiprocessing, allowing many different parallelization schemes depending on the need.



If the spatial propagation is activated, it is necessary to launch the pre-processor that activates the spatial prior sampling before launching the main program.



```

conda activate MuSAenv

python pre_main_spatial.py

python main.py

```



### How to cite

#### MuSA

-  Alonso-González, E., Aalstad, K., Baba, M. W., Revuelto, J., López-Moreno, J. I., Fiddes, J., Essery, R., and Gascoin, S.: The Multiple Snow Data Assimilation System (MuSA v1.0), Geosci. Model Dev., 15, 9127–9155, https://doi.org/10.5194/gmd-15-9127-2022, 2022. 

#### FSM2

-  Mazzotti, G., Essery, R., Moeser, C. D., and Jonas, T.: Resolving small-scale forest snow patterns using an energy balance snow model with a one-layer canopy. Water Resour. Res., 56, https://doi.org/10.1029/2019WR026129, 2020.

-  Essery, R.: A factorial snowpack model (FSM 1.0), Geosci. Model Dev., 8, 3867–3876, https://doi.org/10.5194/gmd-8-3867-2015, 2015. 

#### Related references

- Alonso-González, E., Aalstad, K., Pirk, N., Mazzolini, M., Treichler, D., Leclercq, P., Westermann, S., López-Moreno, J. I., and Gascoin, S.: Spatio-temporal information propagation using sparse observations in hyper-resolution ensemble-based snow data assimilation, Hydrol. Earth Syst. Sci., 27, 4637–4659, https://doi.org/10.5194/hess-27-4637-2023, 2023. 

- Alonso-González, E., Gascoin, S., Arioli, S., and Picard, G.: Exploring the potential of thermal infrared remote sensing to improve a snowpack model through an observing system simulation experiment, EGUsphere The Cryosphere, 17, 3329–3342, https://doi.org/10.5194/tc-17-3329-2023, 2023.

- Alonso-González, E., Gutmann, E., Aalstad, K., Fayad, A., Bouchet, M., and Gascoin, S.: Snowpack dynamics in the Lebanese mountains from quasi-dynamically downscaled ERA5 reanalysis updated by assimilating remotely sensed fractional snow-covered area, Hydrol. Earth Syst. Sci., 25, 4455–4471, https://doi.org/10.5194/hess-25-4455-2021, 2021.

- Fiddes, J., Aalstad, K., and Westermann, S.: Hyper-resolution ensemble-based snow reanalysis in mountain regions using clustering, Hydrol. Earth Syst. Sci., 23, 4717–4736, https://doi.org/10.5194/hess-23-4717-2019, 2019. 

- Aalstad, K., Westermann, S., Schuler, T. V., Boike, J., and Bertino, L.: Ensemble-based assimilation of fractional snow-covered area satellite retrievals to estimate the snow distribution at Arctic sites, The Cryosphere, 12, 247–270, https://doi.org/10.5194/tc-12-247-2018, 2018.