Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/njtierney/ozviridis

Demonstrate the process of improving BoM heatmaps
https://github.com/njtierney/ozviridis

Last synced: about 2 months ago
JSON representation

Demonstrate the process of improving BoM heatmaps

Awesome Lists containing this project

README 

        
---

output:

  md_document:

    variant: markdown_github

---



There's a heatwave in Australia at the moment. And this is the heatmap that is getting shown of Australia:



```{r out.width = "50%", fig.align='center'}



knitr::include_graphics("bom-heat-map.png")



```



Which shows that things are really hot.



But it's also pretty darn ugly. 



I wanted to see if I could make it better, using awesome packages like `viridis` to colour the heat more appropriately.



So this repository is me documenting my struggle with shapefiles and geospatial data. I haven't had to work with this sort of data before, so I'm making an effort to be as transparent as possible with my thoughts on doing this.



Eventually this will get written up into a blog post. But I figured that it would be nice to put on github like this, so that others can contribute, if they like.



OK, so first we load the packages.



```{r pkg-load}



library(rgdal)

library(rgeos)

library(maptools)

library(ggplot2)

library(viridis)



```



Then we load the data, retrieved from [the bom site](http://www.bom.gov.au/jsp/awap/temp/index.jsp) - thanks to [Robbi Bishop Taylor](https://twitter.com/robbibt) for pointing out where to get it!



```{r pkg-read}

library(ozviridis)

data("oz_heat")



class(oz_heat)

# oz_heat <- readGDAL("2017-02-11-oz-heat.grid") # reads in the whole raster



```



# Quick methods for plotting



OK, so I can get a pretty basic plot with base `image`.



```{r}



image(oz_heat) # does a plot



```



But the colour scale isn't great.



With the `viridis` package I can make them look pretty.



```{r}



image(oz_heat,

      col = viridis(15,

                    option = "magma")

      )



image(oz_heat,

      col = viridis(15,

                    option = "plasma")

      )



image(oz_heat,

      col = viridis(15,

                    option = "viridis")

      )



```



There's also the default `spplot` from `sp`



```{r}



spplot(oz_heat,

       col = viridis(15,

                     option = "plasma"))



```



So, the goal from here is to:



- Add the shapefile of Australia

- use `sf`

- Plot using ggplot

- add better legends etc to make it look more similar to the BoM image



## Adding the shapefile of Australia



ROpenSciLabs has a `naturalearthdata` package that is fast and easy to use. Thank you to adamhsparks for finding this and adding to the README!



```{r}

# if (!require("devtools")) install.packages("devtools")

# devtools::install_github("ropenscilabs/rnaturalearth")



library("rnaturalearth")



oz_shape <- rnaturalearth::ne_states(geounit = "australia")



sp::plot(oz_shape)



```



## Using `sf`



We can convert this to a simple features with `st_as_sf`:



```{r}



# convert to simple features

oz_shape_sf <- sf::st_as_sf(oz_shape)



head(oz_shape_sf)



```



## Plot using ggplot2



You can plot this directly from the spatial data, using the following method as described in the tidyverse here, https://github.com/tidyverse/ggplot2/wiki/plotting-polygon-shapefiles:



```{r}



ggplot(oz_shape) +

   aes(x = long, 

       y = lat, 

       group = group) + 

     geom_polygon() +

     geom_path(color = "white") +

     coord_equal()



```



But there are some ways to plot this using the new package `sf`. Do add the temperatures, I think I'll need to add this for each row of each feature.



```{r}

nc <- sf::st_read(system.file("shape/nc.shp", package = "sf"), quiet = TRUE)



head(nc)

```



However, the `ggplot2::geom_sf` unfortunately gives an error when plotting.



```{r eval = FALSE}



ggplot(nc) +

  geom_sf(aes(fill = AREA)) 

# Error in sign(x) : non-numeric argument to mathematical function

```



So where to from here?



Perhaps back to base?



```{r}



sp::plot(oz_shape)



spplot(oz_heat)



```



I feel like there should be some way for me to just write something like:



```{r eval = FALSE}



sp::plot(oz_shape,

         fill = oz_heat)



```



But then this makes me start thinking how I'm used to with ggplot2 - where everything is in a dataframe. Which means that, given that my `sf` dataframe is 11 rows long:



```{r}



tibble::as_tibble(oz_shape_sf)



```



Each row in an `sf` dataframe describes a polygon, and then there are associated values for each of those polygons. So, in order for me to plot the associated colours with each feature, there should then be some interpolation process going on to give each polygon some sort of spatially smoothed temperature value.



This is almost certainly my naïvety showing, but all I really want, is to plot the shape file, and then overlay the appropriately smoothed plot. Sort of like how one can do:



```{r}

plot(cars)

lines(lowess(cars))

```



I'd like to do something like



```{r eval = FALSE}



plot(oz_shape)

colours(oz_heat)



```



But I guess the problem here is how to combine those two pieces of information of different size. 



There is not clear way to go from mapping the colours on a grid, to the shapefile, and what comes after this will be some sort of interpolation step to the shapefiles and polygons.



Or maybe I'm completely wrong about this. And maybe I'm overthinking it and just need to do some proper reading in the right place.



## Bob Rudis's answer on SO



Munging the code from [Bob Rudis's answer on Stack Overflow](http://stackoverflow.com/a/33234951/3764040) about using ggplot2 to plot a raster I have something pretty close to what I'm after.



```{r}

library(rasterVis)

library(ggthemes)

class(oz_shape)

test <- raster(oz_heat)

test_spdf <- as(test, "SpatialPixelsDataFrame")

test_df <- as.data.frame(test_spdf)

colnames(test_df) <- c("value", "x", "y")



```



So here, we specify the raster tiles of the heat information, along with the polygon from the map, and then provide the scale using the Inferno option.



```{r}

oz_heat_map <-

ggplot() +  

  geom_tile(data = test_df, 

            aes(x = x, 

                y = y, 

                fill = value)) + 

  geom_polygon(data = oz_shape, 

               aes(x = long, 

                   y = lat, 

                   group = group), 

               fill = NA, 

               color = "grey10", 

               size = 0.3) +

  scale_fill_viridis(option = "inferno") +

  coord_equal() +

  theme_map() +

  theme(legend.position = c(0.95, 0.5))

  # theme(legend.key.width=unit(2, "cm"))

```



```{r}

oz_heat_map

```



I owe Bob a beer.



My friend Maëlle also pointed me to [Bob's blog post](https://rud.is/b/2016/07/27/u-s-drought-animations-with-the-witchs-brew-purrr-broom-magick/) that does a bit more of what I want, I think.



## Add better legends etc to make it look more similar to the BoM image



Getting there! But for another day.



Things that I want to do before I write this up as a blog post:



- Control the level of smoothing of the tiles, perhaps bin up the temperatures in 5 degree bins.

- Make sure that this approach I'm taking is inline with the current state of the art - should I be using `sf`, can I use `geom_sf`? And more.