DimensionalData.jl

Appearance

321

Dimensional broadcasts with @d and broadcast_dims ​

Broadcasting over AbstractDimArray works as usual with Base Julia broadcasts, except that dimensions are checked for compatibility with each other, and that values match. Strict checks can be turned off globally with strict_broadcast!(false). To avoid even dimension name checks, broadcast over parent(dimarray).

The @d macro is a dimension-aware extension to regular dot broadcasting. broadcast_dims is analogous to Base Julia's broadcast.

Because we know the names of the dimensions, there is no ambiguity in which ones we mean to broadcast together. This means we can permute and reshape dims so that broadcasts that would fail with a regular Array just work with a DimArray.

As an added bonus, broadcast_dims even works on DimStacks. Currently, @d does not work on DimStack.

Example: scaling along the time dimension ​

Define some dimensions:

julia
using DimensionalData
using Dates
using Statistics
julia
julia> x, y, t = X(1:100), Y(1:25), Ti(DateTime(2000):Month(1):DateTime(2000, 12))
(↓ X 1:100,
→ Y 1:25,
↗ Ti DateTime("2000-01-01T00:00:00"):Month(1):DateTime("2000-12-01T00:00:00"))

A DimArray from 1:12 to scale with:

julia
julia> month_scalars = DimArray(month, t)
┌ 12-element DimArray{Int64, 1} month(Ti) ┐
├─────────────────────────────────────────┴────────────────────────────── dims ┐
  ↓ Ti Sampled{DateTime} DateTime("2000-01-01T00:00:00"):Month(1):DateTime("2000-12-01T00:00:00") ForwardOrdered Regular Points
└──────────────────────────────────────────────────────────────────────────────┘
 2000-01-01T00:00:00   1
 2000-02-01T00:00:00   2
 2000-03-01T00:00:00   3
 2000-04-01T00:00:00   4
 2000-05-01T00:00:00   5
 ⋮
 2000-08-01T00:00:00   8
 2000-09-01T00:00:00   9
 2000-10-01T00:00:00  10
 2000-11-01T00:00:00  11
 2000-12-01T00:00:00  12

And a larger DimArray for example data:

julia
julia> data = rand(x, y, t)
┌ 100×25×12 DimArray{Float64, 3} ┐
├────────────────────────────────┴─────────────────────────────────────── dims ┐
  ↓ X Sampled{Int64} 1:100 ForwardOrdered Regular Points,
  → Y Sampled{Int64} 1:25 ForwardOrdered Regular Points,
  ↗ Ti Sampled{DateTime} DateTime("2000-01-01T00:00:00"):Month(1):DateTime("2000-12-01T00:00:00") ForwardOrdered Regular Points
└──────────────────────────────────────────────────────────────────────────────┘
[:, :, 1]
   ↓ →  1         2          3         …  23          24           25
   1    0.904766  0.975141   0.512696      0.38645     0.469594     0.894942
   2    0.880073  0.300648   0.935499      0.919591    0.0802773    0.931704
   3    0.784441  0.106036   0.702299      0.0231189   0.621484     0.880151
   ⋮                                   ⋱                            ⋮
  97    0.535713  0.763233   0.161791      0.396396    0.762295     0.581125
  98    0.199529  0.55822    0.162329      0.767826    0.549098     0.184161
  99    0.25428   0.764507   0.52495       0.934466    0.257629     0.123436
 100    0.403596  0.0441427  0.605193  …   0.904956    0.00843956   0.0959529

A regular broadcast fails:

julia
julia> scaled = data .* month_scalars
ERROR: DimensionMismatch: arrays could not be broadcast to a common size: a has axes X(Base.OneTo(100)) and b has axes Ti(Base.OneTo(12))

But @d knows to broadcast over the Ti dimension:

julia
julia> scaled = @d data .* month_scalars
┌ 100×25×12 DimArray{Float64, 3} ┐
├────────────────────────────────┴─────────────────────────────────────── dims ┐
  ↓ X Sampled{Int64} 1:100 ForwardOrdered Regular Points,
  → Y Sampled{Int64} 1:25 ForwardOrdered Regular Points,
  ↗ Ti Sampled{DateTime} DateTime("2000-01-01T00:00:00"):Month(1):DateTime("2000-12-01T00:00:00") ForwardOrdered Regular Points
└──────────────────────────────────────────────────────────────────────────────┘
[:, :, 1]
   ↓ →  1         2          3         …  23          24           25
   1    0.904766  0.975141   0.512696      0.38645     0.469594     0.894942
   2    0.880073  0.300648   0.935499      0.919591    0.0802773    0.931704
   3    0.784441  0.106036   0.702299      0.0231189   0.621484     0.880151
   ⋮                                   ⋱                            ⋮
  97    0.535713  0.763233   0.161791      0.396396    0.762295     0.581125
  98    0.199529  0.55822    0.162329      0.767826    0.549098     0.184161
  99    0.25428   0.764507   0.52495       0.934466    0.257629     0.123436
 100    0.403596  0.0441427  0.605193  …   0.904956    0.00843956   0.0959529

We can see the means of each month are scaled by the broadcast :

julia
julia> mean(eachslice(data; dims=(X, Y)))
┌ 12-element DimArray{Float64, 1} ┐
├─────────────────────────────────┴────────────────────────────────────── dims ┐
  ↓ Ti Sampled{DateTime} DateTime("2000-01-01T00:00:00"):Month(1):DateTime("2000-12-01T00:00:00") ForwardOrdered Regular Points
└──────────────────────────────────────────────────────────────────────────────┘
 2000-01-01T00:00:00  0.494173
 2000-02-01T00:00:00  0.500948
 2000-03-01T00:00:00  0.508908
 2000-04-01T00:00:00  0.494643
 2000-05-01T00:00:00  0.500352
 ⋮
 2000-08-01T00:00:00  0.491867
 2000-09-01T00:00:00  0.504559
 2000-10-01T00:00:00  0.508242
 2000-11-01T00:00:00  0.503655
 2000-12-01T00:00:00  0.503938
julia
julia> mean(eachslice(scaled; dims=(X, Y)))
┌ 12-element DimArray{Float64, 1} ┐
├─────────────────────────────────┴────────────────────────────────────── dims ┐
  ↓ Ti Sampled{DateTime} DateTime("2000-01-01T00:00:00"):Month(1):DateTime("2000-12-01T00:00:00") ForwardOrdered Regular Points
└──────────────────────────────────────────────────────────────────────────────┘
 2000-01-01T00:00:00  0.494173
 2000-02-01T00:00:00  1.0019
 2000-03-01T00:00:00  1.52673
 2000-04-01T00:00:00  1.97857
 2000-05-01T00:00:00  2.50176
 ⋮
 2000-08-01T00:00:00  3.93493
 2000-09-01T00:00:00  4.54103
 2000-10-01T00:00:00  5.08242
 2000-11-01T00:00:00  5.54021
 2000-12-01T00:00:00  6.04725

You can also use broadcast_dims the same way:

julia
julia> broadcast_dims(*, data, month_scalars)
┌ 100×25×12 DimArray{Float64, 3} ┐
├────────────────────────────────┴─────────────────────────────────────── dims ┐
  ↓ X Sampled{Int64} 1:100 ForwardOrdered Regular Points,
  → Y Sampled{Int64} 1:25 ForwardOrdered Regular Points,
  ↗ Ti Sampled{DateTime} DateTime("2000-01-01T00:00:00"):Month(1):DateTime("2000-12-01T00:00:00") ForwardOrdered Regular Points
└──────────────────────────────────────────────────────────────────────────────┘
[:, :, 1]
   ↓ →  1         2          3         …  23          24           25
   1    0.904766  0.975141   0.512696      0.38645     0.469594     0.894942
   2    0.880073  0.300648   0.935499      0.919591    0.0802773    0.931704
   3    0.784441  0.106036   0.702299      0.0231189   0.621484     0.880151
   ⋮                                   ⋱                            ⋮
  97    0.535713  0.763233   0.161791      0.396396    0.762295     0.581125
  98    0.199529  0.55822    0.162329      0.767826    0.549098     0.184161
  99    0.25428   0.764507   0.52495       0.934466    0.257629     0.123436
 100    0.403596  0.0441427  0.605193  …   0.904956    0.00843956   0.0959529

And with the @d macro you can set the dimension order and other properties of the output array, by passing a single assignment or a NamedTuple argument to @d after the broadcast:

julia
julia> @d data .* month_scalars dims=(Ti, X, Y)
┌ 12×100×25 DimArray{Float64, 3} ┐
├────────────────────────────────┴─────────────────────────────────────── dims ┐
  ↓ Ti Sampled{DateTime} DateTime("2000-01-01T00:00:00"):Month(1):DateTime("2000-12-01T00:00:00") ForwardOrdered Regular Points,
  → X Sampled{Int64} 1:100 ForwardOrdered Regular Points,
  ↗ Y Sampled{Int64} 1:25 ForwardOrdered Regular Points
└──────────────────────────────────────────────────────────────────────────────┘
[:, :, 1]
 ↓ →                   1         …  98         99         100
  2000-01-01T00:00:00  0.904766      0.199529   0.25428     0.403596
  2000-02-01T00:00:00  1.83367       1.87457    1.74266     1.64064
  2000-03-01T00:00:00  0.909957      2.73876    1.49211     0.588845
 ⋮                               ⋱                          ⋮
  2000-09-01T00:00:00  2.95149       0.176132   7.52763     7.90821
  2000-10-01T00:00:00  6.73804   …   7.29753    8.96654     6.02134
  2000-11-01T00:00:00  0.603547      8.54128    8.2705      7.43496
  2000-12-01T00:00:00  2.69051       2.61915    0.713915    0.561869

Or

julia
julia> @d data .* month_scalars (dims=(Ti, X, Y), name=:scaled)
┌ 12×100×25 DimArray{Float64, 3} scaled ┐
├───────────────────────────────────────┴──────────────────────────────── dims ┐
  ↓ Ti Sampled{DateTime} DateTime("2000-01-01T00:00:00"):Month(1):DateTime("2000-12-01T00:00:00") ForwardOrdered Regular Points,
  → X Sampled{Int64} 1:100 ForwardOrdered Regular Points,
  ↗ Y Sampled{Int64} 1:25 ForwardOrdered Regular Points
└──────────────────────────────────────────────────────────────────────────────┘
[:, :, 1]
 ↓ →                   1         …  98         99         100
  2000-01-01T00:00:00  0.904766      0.199529   0.25428     0.403596
  2000-02-01T00:00:00  1.83367       1.87457    1.74266     1.64064
  2000-03-01T00:00:00  0.909957      2.73876    1.49211     0.588845
 ⋮                               ⋱                          ⋮
  2000-09-01T00:00:00  2.95149       0.176132   7.52763     7.90821
  2000-10-01T00:00:00  6.73804   …   7.29753    8.96654     6.02134
  2000-11-01T00:00:00  0.603547      8.54128    8.2705      7.43496
  2000-12-01T00:00:00  2.69051       2.61915    0.713915    0.561869