Rasters.jl

Geospatial raster data reading, writing and manipulation

Rafael Schouten

rafaelschouten@gmail.com

Globe Intstitute, Copenhagen University

2024-07-10

What is a raster?

like an image, but not RGB
values of some variable accross a gridded space
usually has X/Y spatial dimensions
- (e.g. lattitude/longitude)
with a coordinate reference system

may be collected in a dataset with multiple variables
may have more dimensions, like time

DimensionalData.jl integration

Rasters extends DimensionalData.jl
- Raster <: AbstractDimArray and RasterStack <: AbstractDimStack
This gives is the foundation for spatial work
Rasters adds:
- coordinate reference systems
- missing value handling
- File IO
- GIS tools

File Read/Write Backends

File types	Package
Netcdf/hdf5	NCDatasets.jl
Grib (read only)	GRIBDatasets.jl
Zarr (PR nearly done!)	ZarrDatasets.jl
grd (simple Mmap data from R)	native
GeoTIFF and everything else	ArchGDAL.jl

Backend detection

Backend detected in constructors:

# Single array
rast = Raster("myraster.tif")    # Will use ArchGDAL.jl
rast = Raster("myraster.nc")     # Will use NCDatasets

# Multi-array
st = RasterStack("mystack.nc")   # Will use NCDatasets.jl
st = RasterStack("mystack.grib") # Will use GRIBDatasets.jl

DiskArrays.jl integration

For larger-than-memory data

Lazy loading

rast = Raster(filename; lazy=true)

Still lazy after broadcasts:

rast10 = rast .* 10

Reads from disk/network only on getindex:

rast10[X=100 .. 135, Y=20 .. 40]

Chunk patterns

For more efficient lazy reads:

write("rechunked.tif", mem_rast; chunks=(X(256), Y(256)))

RasterDataSources.jl integration

"."

Load a raster from RasterDataSources.jl filename:

using Rasters, RasterDataSources, ArchGDAL
bioclim_filename = RasterDataSources.getraster(WorldClim{BioClim}, 5)
bioclim5 = Raster(bioclim_filename);

[ Info: Starting download for https://geodata.ucdavis.edu/climate/worldclim/2_1/base/wc2.1_10m_bio.zip
┌ Info: Downloading
│   source = "https://geodata.ucdavis.edu/climate/worldclim/2_1/base/wc2.1_10m_bio.zip"
│   dest = "./WorldClim/BioClim/zips/wc2.1_10m_bio.zip"
│   progress = 0.3885
│   time_taken = "1.0 s"
│   time_remaining = "1.58 s"
│   average_speed = "18.460 MiB/s"
│   downloaded = "18.478 MiB"
│   remaining = "29.081 MiB"
└   total = "47.559 MiB"
┌ Info: Downloading
│   source = "https://geodata.ucdavis.edu/climate/worldclim/2_1/base/wc2.1_10m_bio.zip"
│   dest = "./WorldClim/BioClim/zips/wc2.1_10m_bio.zip"
│   progress = 1.0
│   time_taken = "1.89 s"
│   time_remaining = "0.0 s"
│   average_speed = "25.110 MiB/s"
│   downloaded = "47.559 MiB"
│   remaining = "0 bytes"
└   total = "47.559 MiB"

Or use RasterDataSources.jl syntax directly:

bioclim_filename = Raster(WorldClim{BioClim}, 5);

Plotting

Always the right way up!

Plots.jl

using Plots
Plots.plot(bioclim5)

Makie.jl

using CairoMakie
Makie.plot(bioclim5)

GeoMakie.jl

using GeoMakie
fig = Figure();
ga = GeoAxis(fig[1, 1]; dest="+proj=ortho +lon_0=19 +lat_0=72")
Makie.heatmap!(ga, bioclim5)
fig

Common GIS methods

for working with raster data
for using vector/geometry data with raster data

Native rasterization engine

accepts all GeoInterface.jl geometries
extremely fast + threaded
detailed correctness warnings
consistent behaviour and syntax for:

rasterize
coverage
mask
boolmask/missingmask
zonal

Other common methods

extract
crop/extend
trim
mosaic
aggregate
resample

Examples

using Rasters
using ArchGDAL
using Dates
using DataFrames
using GBIF2
using NaturalEarth
using RasterDataSources

`extract`

Extract climate data at specific points:

clim = RasterStack(WorldClim{BioClim})
occ = GBIF2.occurrence_search("Burramys parvus")
# occ is a table with a `:geometry` column, so this "just works"
extract(clim, occ; name=(:bio1, :bio4, :bio7)) |> DataFrame

20×4 DataFrame

Row	geometry	bio1	bio4	bio7
	Tuple…?	Float32?	Float32?	Float32?
1	missing	missing	missing	missing
2	(148.396, -36.3818)	6.88158	484.889	23.133
3	missing	missing	missing	missing
4	(148.333, -36.4333)	7.83572	492.505	23.5028
5	(148.329, -36.4317)	7.83572	492.505	23.5028
6	(148.391, -36.3036)	6.1707	502.798	23.4645
7	missing	missing	missing	missing
8	(147.096, -36.9357)	9.40835	492.567	23.0895
9	(148.347, -36.5047)	8.4207	470.417	23.142
10	missing	missing	missing	missing
11	missing	missing	missing	missing
12	missing	missing	missing	missing
13	(148.241, -36.4001)	7.83572	492.505	23.5028
14	(148.236, -36.5249)	8.0016	479.975	22.8522
15	missing	missing	missing	missing
16	missing	missing	missing	missing
17	missing	missing	missing	missing
18	(147.1, -37.0)	8.25414	485.855	22.2347
19	missing	missing	missing	missing
20	missing	missing	missing	missing

`mask` + `trim`

Mask climate rasters wth country border :

clim = RasterStack(WorldClim{Climate}, (:tmin, :tmax, :prec, :wind); month=July)
countries = naturalearth("ne_10m_admin_0_countries") |> DataFrame
finland = subset(countries, :NAME => ByRow(==("Finland"))).geometry
finland_clim = mask(clim; with=finland) |> trim
Plots.plot(finland_clim; size=(800, 400))

[ Info: Starting download for https://geodata.ucdavis.edu/climate/worldclim/2_1/base/wc2.1_10m_tmin.zip
┌ Info: Downloading
│   source = "https://geodata.ucdavis.edu/climate/worldclim/2_1/base/wc2.1_10m_tmin.zip"
│   dest = "./WorldClim/Climate/zips/wc2.1_10m_tmin.zip"
│   progress = 0.3836
│   time_taken = "1.01 s"
│   time_remaining = "1.62 s"
│   average_speed = "13.606 MiB/s"
│   downloaded = "13.756 MiB"
│   remaining = "22.107 MiB"
└   total = "35.863 MiB"
┌ Info: Downloading
│   source = "https://geodata.ucdavis.edu/climate/worldclim/2_1/base/wc2.1_10m_tmin.zip"
│   dest = "./WorldClim/Climate/zips/wc2.1_10m_tmin.zip"
│   progress = 0.4882
│   time_taken = "2.02 s"
│   time_remaining = "2.11 s"
│   average_speed = "8.681 MiB/s"
│   downloaded = "17.509 MiB"
│   remaining = "18.354 MiB"
└   total = "35.863 MiB"
┌ Info: Downloading
│   source = "https://geodata.ucdavis.edu/climate/worldclim/2_1/base/wc2.1_10m_tmin.zip"
│   dest = "./WorldClim/Climate/zips/wc2.1_10m_tmin.zip"
│   progress = 0.5893
│   time_taken = "3.04 s"
│   time_remaining = "2.12 s"
│   average_speed = "6.963 MiB/s"
│   downloaded = "21.133 MiB"
│   remaining = "14.730 MiB"
└   total = "35.863 MiB"
┌ Info: Downloading
│   source = "https://geodata.ucdavis.edu/climate/worldclim/2_1/base/wc2.1_10m_tmin.zip"
│   dest = "./WorldClim/Climate/zips/wc2.1_10m_tmin.zip"
│   progress = 0.6765
│   time_taken = "4.09 s"
│   time_remaining = "1.95 s"
│   average_speed = "5.936 MiB/s"
│   downloaded = "24.261 MiB"
│   remaining = "11.602 MiB"
└   total = "35.863 MiB"
┌ Info: Downloading
│   source = "https://geodata.ucdavis.edu/climate/worldclim/2_1/base/wc2.1_10m_tmin.zip"
│   dest = "./WorldClim/Climate/zips/wc2.1_10m_tmin.zip"
│   progress = 0.788
│   time_taken = "5.11 s"
│   time_remaining = "1.37 s"
│   average_speed = "5.530 MiB/s"
│   downloaded = "28.259 MiB"
│   remaining = "7.604 MiB"
└   total = "35.863 MiB"
┌ Info: Downloading
│   source = "https://geodata.ucdavis.edu/climate/worldclim/2_1/base/wc2.1_10m_tmin.zip"
│   dest = "./WorldClim/Climate/zips/wc2.1_10m_tmin.zip"
│   progress = 0.8995
│   time_taken = "6.16 s"
│   time_remaining = "0.69 s"
│   average_speed = "5.238 MiB/s"
│   downloaded = "32.257 MiB"
│   remaining = "3.606 MiB"
└   total = "35.863 MiB"
┌ Info: Downloading
│   source = "https://geodata.ucdavis.edu/climate/worldclim/2_1/base/wc2.1_10m_tmin.zip"
│   dest = "./WorldClim/Climate/zips/wc2.1_10m_tmin.zip"
│   progress = 1.0
│   time_taken = "7.12 s"
│   time_remaining = "0.0 s"
│   average_speed = "5.036 MiB/s"
│   downloaded = "35.863 MiB"
│   remaining = "0 bytes"
└   total = "35.863 MiB"
[ Info: Starting download for https://geodata.ucdavis.edu/climate/worldclim/2_1/base/wc2.1_10m_tmax.zip
┌ Info: Downloading
│   source = "https://geodata.ucdavis.edu/climate/worldclim/2_1/base/wc2.1_10m_tmax.zip"
│   dest = "./WorldClim/Climate/zips/wc2.1_10m_tmax.zip"
│   progress = 0.1117
│   time_taken = "1.0 s"
│   time_remaining = "7.97 s"
│   average_speed = "3.978 MiB/s"
│   downloaded = "3.990 MiB"
│   remaining = "31.722 MiB"
└   total = "35.712 MiB"
┌ Info: Downloading
│   source = "https://geodata.ucdavis.edu/climate/worldclim/2_1/base/wc2.1_10m_tmax.zip"
│   dest = "./WorldClim/Climate/zips/wc2.1_10m_tmax.zip"
│   progress = 0.2307
│   time_taken = "2.04 s"
│   time_remaining = "6.8 s"
│   average_speed = "4.039 MiB/s"
│   downloaded = "8.240 MiB"
│   remaining = "27.472 MiB"
└   total = "35.712 MiB"
┌ Info: Downloading
│   source = "https://geodata.ucdavis.edu/climate/worldclim/2_1/base/wc2.1_10m_tmax.zip"
│   dest = "./WorldClim/Climate/zips/wc2.1_10m_tmax.zip"
│   progress = 0.3356
│   time_taken = "3.05 s"
│   time_remaining = "6.04 s"
│   average_speed = "3.930 MiB/s"
│   downloaded = "11.986 MiB"
│   remaining = "23.726 MiB"
└   total = "35.712 MiB"
┌ Info: Downloading
│   source = "https://geodata.ucdavis.edu/climate/worldclim/2_1/base/wc2.1_10m_tmax.zip"
│   dest = "./WorldClim/Climate/zips/wc2.1_10m_tmax.zip"
│   progress = 0.4497
│   time_taken = "4.11 s"
│   time_remaining = "5.02 s"
│   average_speed = "3.912 MiB/s"
│   downloaded = "16.060 MiB"
│   remaining = "19.652 MiB"
└   total = "35.712 MiB"
┌ Info: Downloading
│   source = "https://geodata.ucdavis.edu/climate/worldclim/2_1/base/wc2.1_10m_tmax.zip"
│   dest = "./WorldClim/Climate/zips/wc2.1_10m_tmax.zip"
│   progress = 0.5512
│   time_taken = "5.12 s"
│   time_remaining = "4.17 s"
│   average_speed = "3.843 MiB/s"
│   downloaded = "19.684 MiB"
│   remaining = "16.028 MiB"
└   total = "35.712 MiB"
┌ Info: Downloading
│   source = "https://geodata.ucdavis.edu/climate/worldclim/2_1/base/wc2.1_10m_tmax.zip"
│   dest = "./WorldClim/Climate/zips/wc2.1_10m_tmax.zip"
│   progress = 0.6631
│   time_taken = "6.15 s"
│   time_remaining = "3.12 s"
│   average_speed = "3.851 MiB/s"
│   downloaded = "23.682 MiB"
│   remaining = "12.030 MiB"
└   total = "35.712 MiB"
┌ Info: Downloading
│   source = "https://geodata.ucdavis.edu/climate/worldclim/2_1/base/wc2.1_10m_tmax.zip"
│   dest = "./WorldClim/Climate/zips/wc2.1_10m_tmax.zip"
│   progress = 0.7612
│   time_taken = "7.2 s"
│   time_remaining = "2.26 s"
│   average_speed = "3.774 MiB/s"
│   downloaded = "27.183 MiB"
│   remaining = "8.528 MiB"
└   total = "35.712 MiB"
┌ Info: Downloading
│   source = "https://geodata.ucdavis.edu/climate/worldclim/2_1/base/wc2.1_10m_tmax.zip"
│   dest = "./WorldClim/Climate/zips/wc2.1_10m_tmax.zip"
│   progress = 0.8663
│   time_taken = "8.21 s"
│   time_remaining = "1.27 s"
│   average_speed = "3.770 MiB/s"
│   downloaded = "30.937 MiB"
│   remaining = "4.775 MiB"
└   total = "35.712 MiB"
┌ Info: Downloading
│   source = "https://geodata.ucdavis.edu/climate/worldclim/2_1/base/wc2.1_10m_tmax.zip"
│   dest = "./WorldClim/Climate/zips/wc2.1_10m_tmax.zip"
│   progress = 0.9573
│   time_taken = "9.28 s"
│   time_remaining = "0.41 s"
│   average_speed = "3.684 MiB/s"
│   downloaded = "34.187 MiB"
│   remaining = "1.524 MiB"
└   total = "35.712 MiB"
┌ Info: Downloading
│   source = "https://geodata.ucdavis.edu/climate/worldclim/2_1/base/wc2.1_10m_tmax.zip"
│   dest = "./WorldClim/Climate/zips/wc2.1_10m_tmax.zip"
│   progress = 1.0
│   time_taken = "9.93 s"
│   time_remaining = "0.0 s"
│   average_speed = "3.597 MiB/s"
│   downloaded = "35.712 MiB"
│   remaining = "0 bytes"
└   total = "35.712 MiB"
[ Info: Starting download for https://geodata.ucdavis.edu/climate/worldclim/2_1/base/wc2.1_10m_prec.zip
┌ Info: Downloading
│   source = "https://geodata.ucdavis.edu/climate/worldclim/2_1/base/wc2.1_10m_prec.zip"
│   dest = "./WorldClim/Climate/zips/wc2.1_10m_prec.zip"
│   progress = 0.4133
│   time_taken = "1.04 s"
│   time_remaining = "1.47 s"
│   average_speed = "2.762 MiB/s"
│   downloaded = "2.861 MiB"
│   remaining = "4.061 MiB"
└   total = "6.922 MiB"
┌ Info: Downloading
│   source = "https://geodata.ucdavis.edu/climate/worldclim/2_1/base/wc2.1_10m_prec.zip"
│   dest = "./WorldClim/Climate/zips/wc2.1_10m_prec.zip"
│   progress = 0.9622
│   time_taken = "2.07 s"
│   time_remaining = "0.08 s"
│   average_speed = "3.214 MiB/s"
│   downloaded = "6.660 MiB"
│   remaining = "268.153 KiB"
└   total = "6.922 MiB"
┌ Info: Downloading
│   source = "https://geodata.ucdavis.edu/climate/worldclim/2_1/base/wc2.1_10m_prec.zip"
│   dest = "./WorldClim/Climate/zips/wc2.1_10m_prec.zip"
│   progress = 1.0
│   time_taken = "2.14 s"
│   time_remaining = "0.0 s"
│   average_speed = "3.238 MiB/s"
│   downloaded = "6.922 MiB"
│   remaining = "0 bytes"
└   total = "6.922 MiB"
[ Info: Starting download for https://geodata.ucdavis.edu/climate/worldclim/2_1/base/wc2.1_10m_wind.zip
┌ Info: Downloading
│   source = "https://geodata.ucdavis.edu/climate/worldclim/2_1/base/wc2.1_10m_wind.zip"
│   dest = "./WorldClim/Climate/zips/wc2.1_10m_wind.zip"
│   progress = 0.1272
│   time_taken = "1.01 s"
│   time_remaining = "6.92 s"
│   average_speed = "3.350 MiB/s"
│   downloaded = "3.380 MiB"
│   remaining = "23.185 MiB"
└   total = "26.564 MiB"
┌ Info: Downloading
│   source = "https://geodata.ucdavis.edu/climate/worldclim/2_1/base/wc2.1_10m_wind.zip"
│   dest = "./WorldClim/Climate/zips/wc2.1_10m_wind.zip"
│   progress = 0.2401
│   time_taken = "2.04 s"
│   time_remaining = "6.47 s"
│   average_speed = "3.120 MiB/s"
│   downloaded = "6.378 MiB"
│   remaining = "20.186 MiB"
└   total = "26.564 MiB"
┌ Info: Downloading
│   source = "https://geodata.ucdavis.edu/climate/worldclim/2_1/base/wc2.1_10m_wind.zip"
│   dest = "./WorldClim/Climate/zips/wc2.1_10m_wind.zip"
│   progress = 0.3952
│   time_taken = "3.04 s"
│   time_remaining = "4.66 s"
│   average_speed = "3.448 MiB/s"
│   downloaded = "10.498 MiB"
│   remaining = "16.066 MiB"
└   total = "26.564 MiB"
┌ Info: Downloading
│   source = "https://geodata.ucdavis.edu/climate/worldclim/2_1/base/wc2.1_10m_wind.zip"
│   dest = "./WorldClim/Climate/zips/wc2.1_10m_wind.zip"
│   progress = 0.5129
│   time_taken = "4.07 s"
│   time_remaining = "3.86 s"
│   average_speed = "3.350 MiB/s"
│   downloaded = "13.626 MiB"
│   remaining = "12.938 MiB"
└   total = "26.564 MiB"
┌ Info: Downloading
│   source = "https://geodata.ucdavis.edu/climate/worldclim/2_1/base/wc2.1_10m_wind.zip"
│   dest = "./WorldClim/Climate/zips/wc2.1_10m_wind.zip"
│   progress = 0.6307
│   time_taken = "5.08 s"
│   time_remaining = "2.98 s"
│   average_speed = "3.295 MiB/s"
│   downloaded = "16.754 MiB"
│   remaining = "9.810 MiB"
└   total = "26.564 MiB"
┌ Info: Downloading
│   source = "https://geodata.ucdavis.edu/climate/worldclim/2_1/base/wc2.1_10m_wind.zip"
│   dest = "./WorldClim/Climate/zips/wc2.1_10m_wind.zip"
│   progress = 0.7576
│   time_taken = "6.12 s"
│   time_remaining = "1.96 s"
│   average_speed = "3.288 MiB/s"
│   downloaded = "20.126 MiB"
│   remaining = "6.438 MiB"
└   total = "26.564 MiB"
┌ Info: Downloading
│   source = "https://geodata.ucdavis.edu/climate/worldclim/2_1/base/wc2.1_10m_wind.zip"
│   dest = "./WorldClim/Climate/zips/wc2.1_10m_wind.zip"
│   progress = 0.913
│   time_taken = "7.12 s"
│   time_remaining = "0.68 s"
│   average_speed = "3.405 MiB/s"
│   downloaded = "24.254 MiB"
│   remaining = "2.311 MiB"
└   total = "26.564 MiB"
┌ Info: Downloading
│   source = "https://geodata.ucdavis.edu/climate/worldclim/2_1/base/wc2.1_10m_wind.zip"
│   dest = "./WorldClim/Climate/zips/wc2.1_10m_wind.zip"
│   progress = 1.0
│   time_taken = "7.65 s"
│   time_remaining = "0.0 s"
│   average_speed = "3.472 MiB/s"
│   downloaded = "26.564 MiB"
│   remaining = "0 bytes"
└   total = "26.564 MiB"

`zonal` statistics

Find the hottest and coldest countries in July:

clim = Raster(WorldClim{Climate}, :tmax; month=July)
countries = naturalearth("ne_10m_admin_0_countries") |> DataFrame
countries.july_maxtemp = zonal(Statistics.mean, clim; 
    of=countries, boundary=:touches, progress=false
)
filtered = subset(countries, :july_maxtemp => ByRow(!isnan))
sort!(filtered, :july_maxtemp).NAME

254-element Vector{String}:
 "Antarctica"
 "Greenland"
 "Southern Patagonian Ice Field"
 "S. Geo. and the Is."
 "Heard I. and McDonald Is."
 "Falkland Is."
 "Fr. S. Antarctic Lands"
 "Siachen Glacier"
 "Chile"
 "New Zealand"
 ⋮
 "Bir Tawil"
 "Bahrain"
 "Mauritania"
 "Algeria"
 "Qatar"
 "Saudi Arabia"
 "United Arab Emirates"
 "Iraq"
 "Kuwait"

Thanks

Especially to Rasters.jl contributors!

Any problems, make github issues at
https://github.com/rafaqz/Rasters.jl

(Please include all files in a MWE!)