DimensionalData.jl

Appearance

Dimensional broadcasts with @d and broadcast_dims ​

Broadcasting over AbstractDimArray works as usual with Base Julia broadcasts, except that dimensions are checked for compatibility with eachother, and that values match. Strict checks can be turned of globally with strict_broadcast!(false). To avoid even dimension name checks, broadcast over parent(dimarray).

The @d macro is a dimension-aware extension to regular dot brodcasting. broadcast_dims and broadcast_dims are analagous to Base julia broadcast.

Because we know the names of the dimensions, there is no ambiguity in which one we mean to broadcast together. This means we can permute and reshape dims so that broadcasts that would fail with a regular Array just work with a DimArray.

As an added bonus, broadcast_dims even works on DimStacks. Currently @d does not work on DimStack.

Example: scaling along the time dimension ​

Define some dimensions:

julia
using DimensionalData
using Dates
using Statistics
julia
julia> x, y, t = X(1:100), Y(1:25), Ti(DateTime(2000):Month(1):DateTime(2000, 12))
(↓ X  1:100,
→ Y  1:25,
↗ Ti Dates.DateTime("2000-01-01T00:00:00"):Dates.Month(1):Dates.DateTime("2000-12-01T00:00:00"))

A DimArray from 1:12 to scale with:

julia
julia> month_scalars = DimArray(month, t)
╭────────────────────────────────────────╮
│ 12-element DimArray{Int64,1} month(Ti) │
├────────────────────────────────────────┴─────────────────────────────── dims ┐
  ↓ Ti Sampled{Dates.DateTime} Dates.DateTime("2000-01-01T00:00:00"):Dates.Month(1):Dates.DateTime("2000-12-01T00:00:00") ForwardOrdered Regular Points
└──────────────────────────────────────────────────────────────────────────────┘
 2000-01-01T00:00:00   1
 2000-02-01T00:00:00   2
 2000-03-01T00:00:00   3
 2000-04-01T00:00:00   4
 2000-05-01T00:00:00   5
 2000-06-01T00:00:00   6
 2000-07-01T00:00:00   7
 2000-08-01T00:00:00   8
 2000-09-01T00:00:00   9
 2000-10-01T00:00:00  10
 2000-11-01T00:00:00  11
 2000-12-01T00:00:00  12

And a larger DimArray for example data:

julia
julia> data = rand(x, y, t)
╭───────────────────────────────╮
│ 100×25×12 DimArray{Float64,3} │
├───────────────────────────────┴──────────────────────────────────────── dims ┐
  ↓ X  Sampled{Int64} 1:100 ForwardOrdered Regular Points,
  → Y  Sampled{Int64} 1:25 ForwardOrdered Regular Points,
  ↗ Ti Sampled{Dates.DateTime} Dates.DateTime("2000-01-01T00:00:00"):Dates.Month(1):Dates.DateTime("2000-12-01T00:00:00") ForwardOrdered Regular Points
└──────────────────────────────────────────────────────────────────────────────┘
[:, :, 1]
   ↓ →  1          2         3         …  23          24          25
   1    0.792885   0.896315  0.28708       0.818644    0.568131    0.967865
   2    0.0959455  0.164724  0.491767      0.515061    0.0416986   0.31065
   3    0.23546    0.652353  0.201734      0.568302    0.875221    0.651784
   4    0.2344     0.390033  0.108592      0.69635     0.655684    0.564589
   ⋮                                   ⋱                           ⋮
  97    0.192183   0.76163   0.924417      0.0502944   0.736214    0.959028
  98    0.204551   0.725738  0.511086      0.566557    0.268319    0.790283
  99    0.250752   0.612779  0.974596      0.246796    0.555456    0.27181
 100    0.295548   0.364488  0.21714   …   0.687223    0.068913    0.453973

A regular broadcast fails:

julia
julia> scaled = data .* month_scalars
ERROR: DimensionMismatch: arrays could not be broadcast to a common size; got a dimension with lengths 100 and 12

But @d knows to broadcast over the Ti dimension:

julia
julia> scaled = @d data .* month_scalars
╭───────────────────────────────╮
│ 100×25×12 DimArray{Float64,3} │
├───────────────────────────────┴──────────────────────────────────────── dims ┐
  ↓ X  Sampled{Int64} 1:100 ForwardOrdered Regular Points,
  → Y  Sampled{Int64} 1:25 ForwardOrdered Regular Points,
  ↗ Ti Sampled{Dates.DateTime} Dates.DateTime("2000-01-01T00:00:00"):Dates.Month(1):Dates.DateTime("2000-12-01T00:00:00") ForwardOrdered Regular Points
└──────────────────────────────────────────────────────────────────────────────┘
[:, :, 1]
   ↓ →  1          2         3         …  23          24          25
   1    0.792885   0.896315  0.28708       0.818644    0.568131    0.967865
   2    0.0959455  0.164724  0.491767      0.515061    0.0416986   0.31065
   3    0.23546    0.652353  0.201734      0.568302    0.875221    0.651784
   4    0.2344     0.390033  0.108592      0.69635     0.655684    0.564589
   ⋮                                   ⋱                           ⋮
  97    0.192183   0.76163   0.924417      0.0502944   0.736214    0.959028
  98    0.204551   0.725738  0.511086      0.566557    0.268319    0.790283
  99    0.250752   0.612779  0.974596      0.246796    0.555456    0.27181
 100    0.295548   0.364488  0.21714   …   0.687223    0.068913    0.453973

We can see the means of each month are scaled by the broadcast :

julia
julia> mean(eachslice(data; dims=(X, Y)))
╭────────────────────────────────╮
│ 12-element DimArray{Float64,1} │
├────────────────────────────────┴─────────────────────────────────────── dims ┐
  ↓ Ti Sampled{Dates.DateTime} Dates.DateTime("2000-01-01T00:00:00"):Dates.Month(1):Dates.DateTime("2000-12-01T00:00:00") ForwardOrdered Regular Points
└──────────────────────────────────────────────────────────────────────────────┘
 2000-01-01T00:00:00  0.496108
 2000-02-01T00:00:00  0.49795
 2000-03-01T00:00:00  0.493708
 2000-04-01T00:00:00  0.492377
 2000-05-01T00:00:00  0.496201
 2000-06-01T00:00:00  0.496665
 2000-07-01T00:00:00  0.489762
 2000-08-01T00:00:00  0.503719
 2000-09-01T00:00:00  0.505946
 2000-10-01T00:00:00  0.499719
 2000-11-01T00:00:00  0.490968
 2000-12-01T00:00:00  0.501068
julia
julia> mean(eachslice(scaled; dims=(X, Y)))
╭────────────────────────────────╮
│ 12-element DimArray{Float64,1} │
├────────────────────────────────┴─────────────────────────────────────── dims ┐
  ↓ Ti Sampled{Dates.DateTime} Dates.DateTime("2000-01-01T00:00:00"):Dates.Month(1):Dates.DateTime("2000-12-01T00:00:00") ForwardOrdered Regular Points
└──────────────────────────────────────────────────────────────────────────────┘
 2000-01-01T00:00:00  0.496108
 2000-02-01T00:00:00  0.995899
 2000-03-01T00:00:00  1.48113
 2000-04-01T00:00:00  1.96951
 2000-05-01T00:00:00  2.48101
 2000-06-01T00:00:00  2.97999
 2000-07-01T00:00:00  3.42833
 2000-08-01T00:00:00  4.02976
 2000-09-01T00:00:00  4.55351
 2000-10-01T00:00:00  4.99719
 2000-11-01T00:00:00  5.40065
 2000-12-01T00:00:00  6.01282

You can also use broadcast_dims the same way:

julia
julia> broadcast_dims(*, data, month_scalars)
╭───────────────────────────────╮
│ 100×25×12 DimArray{Float64,3} │
├───────────────────────────────┴──────────────────────────────────────── dims ┐
  ↓ X  Sampled{Int64} 1:100 ForwardOrdered Regular Points,
  → Y  Sampled{Int64} 1:25 ForwardOrdered Regular Points,
  ↗ Ti Sampled{Dates.DateTime} Dates.DateTime("2000-01-01T00:00:00"):Dates.Month(1):Dates.DateTime("2000-12-01T00:00:00") ForwardOrdered Regular Points
└──────────────────────────────────────────────────────────────────────────────┘
[:, :, 1]
   ↓ →  1          2         3         …  23          24          25
   1    0.792885   0.896315  0.28708       0.818644    0.568131    0.967865
   2    0.0959455  0.164724  0.491767      0.515061    0.0416986   0.31065
   3    0.23546    0.652353  0.201734      0.568302    0.875221    0.651784
   4    0.2344     0.390033  0.108592      0.69635     0.655684    0.564589
   ⋮                                   ⋱                           ⋮
  97    0.192183   0.76163   0.924417      0.0502944   0.736214    0.959028
  98    0.204551   0.725738  0.511086      0.566557    0.268319    0.790283
  99    0.250752   0.612779  0.974596      0.246796    0.555456    0.27181
 100    0.295548   0.364488  0.21714   …   0.687223    0.068913    0.453973

And with the @d macro you can set the dimension order and other properties of the output array, by passing a single assignment or a NamedTuple argument to @d after the broadcast:

julia
julia> @d data .* month_scalars dims=(Ti, X, Y)
╭───────────────────────────────╮
│ 12×100×25 DimArray{Float64,3} │
├───────────────────────────────┴──────────────────────────────────────── dims ┐
  ↓ Ti Sampled{Dates.DateTime} Dates.DateTime("2000-01-01T00:00:00"):Dates.Month(1):Dates.DateTime("2000-12-01T00:00:00") ForwardOrdered Regular Points,
  → X  Sampled{Int64} 1:100 ForwardOrdered Regular Points,
  ↗ Y  Sampled{Int64} 1:25 ForwardOrdered Regular Points
└──────────────────────────────────────────────────────────────────────────────┘
[:, :, 1]
 ↓ →                   1         …  98         99         100
  2000-01-01T00:00:00  0.792885      0.204551   0.250752    0.295548
  2000-02-01T00:00:00  1.27802       0.728611   1.04046     1.10447
  2000-03-01T00:00:00  1.13589       1.54853    0.686287    0.929592
  2000-04-01T00:00:00  3.49029       1.09764    3.2115      1.38795
 ⋮                               ⋱                          ⋮
  2000-09-01T00:00:00  1.89618       0.203703   4.47963     2.55114
  2000-10-01T00:00:00  9.58054   …   5.3793     0.118632    7.12731
  2000-11-01T00:00:00  0.134208      5.08512    6.19995     2.74204
  2000-12-01T00:00:00  7.60111      11.5008    10.2026      6.54483

Or

julia
julia> @d data .* month_scalars (dims=(Ti, X, Y), name=:scaled)
╭──────────────────────────────────────╮
│ 12×100×25 DimArray{Float64,3} scaled │
├──────────────────────────────────────┴───────────────────────────────── dims ┐
  ↓ Ti Sampled{Dates.DateTime} Dates.DateTime("2000-01-01T00:00:00"):Dates.Month(1):Dates.DateTime("2000-12-01T00:00:00") ForwardOrdered Regular Points,
  → X  Sampled{Int64} 1:100 ForwardOrdered Regular Points,
  ↗ Y  Sampled{Int64} 1:25 ForwardOrdered Regular Points
└──────────────────────────────────────────────────────────────────────────────┘
[:, :, 1]
 ↓ →                   1         …  98         99         100
  2000-01-01T00:00:00  0.792885      0.204551   0.250752    0.295548
  2000-02-01T00:00:00  1.27802       0.728611   1.04046     1.10447
  2000-03-01T00:00:00  1.13589       1.54853    0.686287    0.929592
  2000-04-01T00:00:00  3.49029       1.09764    3.2115      1.38795
 ⋮                               ⋱                          ⋮
  2000-09-01T00:00:00  1.89618       0.203703   4.47963     2.55114
  2000-10-01T00:00:00  9.58054   …   5.3793     0.118632    7.12731
  2000-11-01T00:00:00  0.134208      5.08512    6.19995     2.74204
  2000-12-01T00:00:00  7.60111      11.5008    10.2026      6.54483