Dimensional broadcasts with @d
and broadcast_dims
Broadcasting over AbstractDimArray works as usual with Base Julia broadcasts, except that dimensions are checked for compatibility with each other, and that values match. Strict checks can be turned off globally with strict_broadcast!(false)
. To avoid even dimension name checks, broadcast over parent(dimarray)
.
The @d
macro is a dimension-aware extension to regular dot broadcasting. broadcast_dims
is analogous to Base Julia's broadcast
.
Because we know the names of the dimensions, there is no ambiguity in which ones we mean to broadcast together. This means we can permute and reshape dims so that broadcasts that would fail with a regular Array
just work with a DimArray
.
As an added bonus, broadcast_dims
even works on DimStack
s. Currently, @d
does not work on DimStack
.
Example: scaling along the time dimension
Define some dimensions:
using DimensionalData
using Dates
using Statistics
julia> x, y, t = X(1:100), Y(1:25), Ti(DateTime(2000):Month(1):DateTime(2000, 12))
(↓ X 1:100,
→ Y 1:25,
↗ Ti DateTime("2000-01-01T00:00:00"):Month(1):DateTime("2000-12-01T00:00:00"))
A DimArray from 1:12 to scale with:
julia> month_scalars = DimArray(month, t)
╭─────────────────────────────────────────╮
│ 12-element DimArray{Int64, 1} month(Ti) │
├─────────────────────────────────────────┴────────────────────────────── dims ┐
↓ Ti Sampled{DateTime} DateTime("2000-01-01T00:00:00"):Month(1):DateTime("2000-12-01T00:00:00") ForwardOrdered Regular Points
└──────────────────────────────────────────────────────────────────────────────┘
2000-01-01T00:00:00 1
2000-02-01T00:00:00 2
2000-03-01T00:00:00 3
2000-04-01T00:00:00 4
2000-05-01T00:00:00 5
2000-06-01T00:00:00 6
2000-07-01T00:00:00 7
2000-08-01T00:00:00 8
2000-09-01T00:00:00 9
2000-10-01T00:00:00 10
2000-11-01T00:00:00 11
2000-12-01T00:00:00 12
And a larger DimArray for example data:
julia> data = rand(x, y, t)
╭────────────────────────────────╮
│ 100×25×12 DimArray{Float64, 3} │
├────────────────────────────────┴─────────────────────────────────────── dims ┐
↓ X Sampled{Int64} 1:100 ForwardOrdered Regular Points,
→ Y Sampled{Int64} 1:25 ForwardOrdered Regular Points,
↗ Ti Sampled{DateTime} DateTime("2000-01-01T00:00:00"):Month(1):DateTime("2000-12-01T00:00:00") ForwardOrdered Regular Points
└──────────────────────────────────────────────────────────────────────────────┘
[:, :, 1]
↓ → 1 2 3 … 23 24 25
1 0.0275537 0.171798 0.661454 0.580336 0.826641 0.94561
2 0.455273 0.380872 0.43597 0.312325 0.931262 0.223114
3 0.333692 0.46747 0.618895 0.808742 0.576437 0.657325
4 0.5207 0.95715 0.534996 0.25951 0.877483 0.287422
⋮ ⋱ ⋮
97 0.617939 0.980869 0.338072 0.910816 0.657033 0.523385
98 0.549925 0.340573 0.895484 0.297808 0.518075 0.202221
99 0.335082 0.14166 0.290357 0.393876 0.177009 0.826134
100 0.249064 0.0313839 0.0966582 … 0.857851 0.80082 0.547268
A regular broadcast fails:
julia> scaled = data .* month_scalars
ERROR: DimensionMismatch: arrays could not be broadcast to a common size: a has axes DimensionalData.Dimensions.DimUnitRange(Base.OneTo(100), X{Sampled{Int64, UnitRange{Int64}, ForwardOrdered, Regular{Int64}, Points, NoMetadata}}([1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100])) and b has axes DimensionalData.Dimensions.DimUnitRange(Base.OneTo(12), Ti{Sampled{DateTime, StepRange{DateTime, Month}, ForwardOrdered, Regular{Month}, Points, NoMetadata}}([DateTime("2000-01-01T00:00:00"), DateTime("2000-02-01T00:00:00"), DateTime("2000-03-01T00:00:00"), DateTime("2000-04-01T00:00:00"), DateTime("2000-05-01T00:00:00"), DateTime("2000-06-01T00:00:00"), DateTime("2000-07-01T00:00:00"), DateTime("2000-08-01T00:00:00"), DateTime("2000-09-01T00:00:00"), DateTime("2000-10-01T00:00:00"), DateTime("2000-11-01T00:00:00"), DateTime("2000-12-01T00:00:00")]))
But @d
knows to broadcast over the Ti
dimension:
julia> scaled = @d data .* month_scalars
╭────────────────────────────────╮
│ 100×25×12 DimArray{Float64, 3} │
├────────────────────────────────┴─────────────────────────────────────── dims ┐
↓ X Sampled{Int64} 1:100 ForwardOrdered Regular Points,
→ Y Sampled{Int64} 1:25 ForwardOrdered Regular Points,
↗ Ti Sampled{DateTime} DateTime("2000-01-01T00:00:00"):Month(1):DateTime("2000-12-01T00:00:00") ForwardOrdered Regular Points
└──────────────────────────────────────────────────────────────────────────────┘
[:, :, 1]
↓ → 1 2 3 … 23 24 25
1 0.0275537 0.171798 0.661454 0.580336 0.826641 0.94561
2 0.455273 0.380872 0.43597 0.312325 0.931262 0.223114
3 0.333692 0.46747 0.618895 0.808742 0.576437 0.657325
4 0.5207 0.95715 0.534996 0.25951 0.877483 0.287422
⋮ ⋱ ⋮
97 0.617939 0.980869 0.338072 0.910816 0.657033 0.523385
98 0.549925 0.340573 0.895484 0.297808 0.518075 0.202221
99 0.335082 0.14166 0.290357 0.393876 0.177009 0.826134
100 0.249064 0.0313839 0.0966582 … 0.857851 0.80082 0.547268
We can see the means of each month are scaled by the broadcast :
julia> mean(eachslice(data; dims=(X, Y)))
╭─────────────────────────────────╮
│ 12-element DimArray{Float64, 1} │
├─────────────────────────────────┴────────────────────────────────────── dims ┐
↓ Ti Sampled{DateTime} DateTime("2000-01-01T00:00:00"):Month(1):DateTime("2000-12-01T00:00:00") ForwardOrdered Regular Points
└──────────────────────────────────────────────────────────────────────────────┘
2000-01-01T00:00:00 0.499346
2000-02-01T00:00:00 0.504421
2000-03-01T00:00:00 0.500006
2000-04-01T00:00:00 0.500925
2000-05-01T00:00:00 0.498882
2000-06-01T00:00:00 0.509772
2000-07-01T00:00:00 0.504664
2000-08-01T00:00:00 0.48904
2000-09-01T00:00:00 0.501033
2000-10-01T00:00:00 0.512691
2000-11-01T00:00:00 0.509249
2000-12-01T00:00:00 0.504887
julia> mean(eachslice(scaled; dims=(X, Y)))
╭─────────────────────────────────╮
│ 12-element DimArray{Float64, 1} │
├─────────────────────────────────┴────────────────────────────────────── dims ┐
↓ Ti Sampled{DateTime} DateTime("2000-01-01T00:00:00"):Month(1):DateTime("2000-12-01T00:00:00") ForwardOrdered Regular Points
└──────────────────────────────────────────────────────────────────────────────┘
2000-01-01T00:00:00 0.499346
2000-02-01T00:00:00 1.00884
2000-03-01T00:00:00 1.50002
2000-04-01T00:00:00 2.0037
2000-05-01T00:00:00 2.49441
2000-06-01T00:00:00 3.05863
2000-07-01T00:00:00 3.53265
2000-08-01T00:00:00 3.91232
2000-09-01T00:00:00 4.50929
2000-10-01T00:00:00 5.12691
2000-11-01T00:00:00 5.60174
2000-12-01T00:00:00 6.05865
You can also use broadcast_dims
the same way:
julia> broadcast_dims(*, data, month_scalars)
╭────────────────────────────────╮
│ 100×25×12 DimArray{Float64, 3} │
├────────────────────────────────┴─────────────────────────────────────── dims ┐
↓ X Sampled{Int64} 1:100 ForwardOrdered Regular Points,
→ Y Sampled{Int64} 1:25 ForwardOrdered Regular Points,
↗ Ti Sampled{DateTime} DateTime("2000-01-01T00:00:00"):Month(1):DateTime("2000-12-01T00:00:00") ForwardOrdered Regular Points
└──────────────────────────────────────────────────────────────────────────────┘
[:, :, 1]
↓ → 1 2 3 … 23 24 25
1 0.0275537 0.171798 0.661454 0.580336 0.826641 0.94561
2 0.455273 0.380872 0.43597 0.312325 0.931262 0.223114
3 0.333692 0.46747 0.618895 0.808742 0.576437 0.657325
4 0.5207 0.95715 0.534996 0.25951 0.877483 0.287422
⋮ ⋱ ⋮
97 0.617939 0.980869 0.338072 0.910816 0.657033 0.523385
98 0.549925 0.340573 0.895484 0.297808 0.518075 0.202221
99 0.335082 0.14166 0.290357 0.393876 0.177009 0.826134
100 0.249064 0.0313839 0.0966582 … 0.857851 0.80082 0.547268
And with the @d
macro you can set the dimension order and other properties of the output array, by passing a single assignment or a NamedTuple
argument to @d
after the broadcast:
julia> @d data .* month_scalars dims=(Ti, X, Y)
╭────────────────────────────────╮
│ 12×100×25 DimArray{Float64, 3} │
├────────────────────────────────┴─────────────────────────────────────── dims ┐
↓ Ti Sampled{DateTime} DateTime("2000-01-01T00:00:00"):Month(1):DateTime("2000-12-01T00:00:00") ForwardOrdered Regular Points,
→ X Sampled{Int64} 1:100 ForwardOrdered Regular Points,
↗ Y Sampled{Int64} 1:25 ForwardOrdered Regular Points
└──────────────────────────────────────────────────────────────────────────────┘
[:, :, 1]
↓ → 1 … 98 99 100
2000-01-01T00:00:00 0.0275537 0.549925 0.335082 0.249064
2000-02-01T00:00:00 1.45622 1.01922 0.269022 1.91317
2000-03-01T00:00:00 2.12888 2.95191 1.13754 0.411866
2000-04-01T00:00:00 2.90878 0.952418 2.86682 0.887562
⋮ ⋱ ⋮
2000-09-01T00:00:00 7.06221 6.76357 4.42655 7.54669
2000-10-01T00:00:00 0.524585 … 5.03388 8.99929 1.02435
2000-11-01T00:00:00 5.58339 7.95765 1.30559 9.12414
2000-12-01T00:00:00 6.75149 7.79494 11.3744 2.69071
Or
julia> @d data .* month_scalars (dims=(Ti, X, Y), name=:scaled)
╭───────────────────────────────────────╮
│ 12×100×25 DimArray{Float64, 3} scaled │
├───────────────────────────────────────┴──────────────────────────────── dims ┐
↓ Ti Sampled{DateTime} DateTime("2000-01-01T00:00:00"):Month(1):DateTime("2000-12-01T00:00:00") ForwardOrdered Regular Points,
→ X Sampled{Int64} 1:100 ForwardOrdered Regular Points,
↗ Y Sampled{Int64} 1:25 ForwardOrdered Regular Points
└──────────────────────────────────────────────────────────────────────────────┘
[:, :, 1]
↓ → 1 … 98 99 100
2000-01-01T00:00:00 0.0275537 0.549925 0.335082 0.249064
2000-02-01T00:00:00 1.45622 1.01922 0.269022 1.91317
2000-03-01T00:00:00 2.12888 2.95191 1.13754 0.411866
2000-04-01T00:00:00 2.90878 0.952418 2.86682 0.887562
⋮ ⋱ ⋮
2000-09-01T00:00:00 7.06221 6.76357 4.42655 7.54669
2000-10-01T00:00:00 0.524585 … 5.03388 8.99929 1.02435
2000-11-01T00:00:00 5.58339 7.95765 1.30559 9.12414
2000-12-01T00:00:00 6.75149 7.79494 11.3744 2.69071