DimensionalData.jl

Appearance

Selectors ​

In addition to choosing dimensions by name, we can also select values within them.

First, we can create a DimArray with lookup values as well as dimension names:

julia
using DimensionalData
julia
julia> A = rand(X(1.0:0.2:2.0), Y([:a, :b, :c]))
╭─────────────────────────╮
│ 6×3 DimArray{Float64,2} │
├─────────────────────────┴────────────────────────────────────────────── dims ┐
  ↓ X Sampled{Float64} 1.0:0.2:2.0 ForwardOrdered Regular Points,
  → Y Categorical{Symbol} [:a, :b, :c] ForwardOrdered
└──────────────────────────────────────────────────────────────────────────────┘
 ↓ →   :a        :b        :c
 1.0  0.27736   0.802776  0.621603
 1.2  0.444305  0.156538  0.768488
 1.4  0.184738  0.226064  0.869012
 1.6  0.772277  0.764895  0.101231
 1.8  0.711133  0.86273   0.239921
 2.0  0.883222  0.748041  0.511313

Then we can use the Selector to select values from the array:

The At(x) selector gets the index or indices exactly matching the passed in value(s).

julia
julia> A[X=At(1.2), Y=At(:c)]
0.7684883626668314

Or within a tolerance:

julia
julia> A[X=At(0.99:0.201:1.5; atol=0.05)]
╭─────────────────────────╮
│ 3×3 DimArray{Float64,2} │
├─────────────────────────┴────────────────────────────────────────────── dims ┐
  ↓ X Sampled{Float64} [1.0, 1.2, 1.4] ForwardOrdered Irregular Points,
  → Y Categorical{Symbol} [:a, :b, :c] ForwardOrdered
└──────────────────────────────────────────────────────────────────────────────┘
 ↓ →   :a        :b        :c
 1.0  0.27736   0.802776  0.621603
 1.2  0.444305  0.156538  0.768488
 1.4  0.184738  0.226064  0.869012

At can also take vectors and ranges:

julia
julia> A[X=At(1.2:0.2:1.5), Y=At([:a, :c])]
╭─────────────────────────╮
│ 2×2 DimArray{Float64,2} │
├─────────────────────────┴────────────────────────────────────────────── dims ┐
  ↓ X Sampled{Float64} [1.2, 1.4] ForwardOrdered Irregular Points,
  → Y Categorical{Symbol} [:a, :c] ForwardOrdered
└──────────────────────────────────────────────────────────────────────────────┘
 ↓ →   :a        :c
 1.2  0.444305  0.768488
 1.4  0.184738  0.869012

Lookups ​

Selectors find indices in the Lookup of each dimension. Lookups wrap other AbstractArray (often AbstractRange) but add additional traits to facilitate fast lookups or specifying point or interval behaviour. These are usually detected automatically.

julia
using DimensionalData.Lookups

The Sampled(x) lookup holds values sampled along an axis. They may be Ordered/Unordered, Intervals/Points, and Regular/Irregular.

Most of these properties are usually detected automatically, but here we create a Sampled lookup manually:

julia
julia> l = Sampled(10.0:10.0:100.0; order=ForwardOrdered(), span=Regular(10.0), sampling=Intervals(Start()))
Sampled{Float64} ForwardOrdered Regular Intervals{Start}
wrapping: 10.0:10.0:100.0

To specify Irregular Intervals, we should include the outer bounds of the lookup, as we can't determine them from the vector.

julia
julia> l = Sampled([13, 8, 5, 3, 2, 1]; order=ForwardOrdered(), span=Irregular(1, 21), sampling=Intervals(Start()))
Sampled{Int64} ForwardOrdered Irregular Intervals{Start}
wrapping: 6-element Vector{Int64}:
 13
  8
  5
  3
  2
  1

Lookup autodetection ​

When we define an array, extra properties are detected:

julia
julia> A = DimArray(rand(7, 5), (X(10:10:70), Y([:a, :b, :c, :d, :e])))
╭─────────────────────────╮
│ 7×5 DimArray{Float64,2} │
├─────────────────────────┴────────────────────────────────────────────── dims ┐
  ↓ X Sampled{Int64} 10:10:70 ForwardOrdered Regular Points,
  → Y Categorical{Symbol} [:a, :b, :c, :d, :e] ForwardOrdered
└──────────────────────────────────────────────────────────────────────────────┘
  ↓ →   :a         :b        :c        :d        :e
 10    0.71372    0.301659  0.229418  0.442111  0.680987
 20    0.905616   0.307762  0.228248  0.882656  0.396585
 30    0.0928922  0.651538  0.152068  0.683795  0.687921
 40    0.441181   0.314906  0.650675  0.570603  0.647225
 50    0.621662   0.196478  0.97293   0.929034  0.855976
 60    0.72217    0.791844  0.883323  0.21921   0.0575993
 70    0.896257   0.758149  0.679453  0.506221  0.771237

This array has a Sampled lookup with ForwardOrdered Regular Points for X, and a Categorical ForwardOrdered for Y.

Most lookup types and properties are detected automatically like this from the arrays and ranges used.

  • Arrays and ranges of String, Symbol, and Char are set to Categorical lookup.

    • order is detected as Unordered, ForwardOrdered, or ReverseOrdered

  • Arrays and ranges of Number, DateTime, and other things are set to Sampled lookups.

    • order is detected as Unordered, ForwardOrdered, or ReverseOrdered.

    • sampling is set to Points() unless the values are IntervalSets.Interval, then Intervals(Center()) is used.

    • span is detected as Regular(step(range)) for AbstractRange and Irregular(nothing, nothing) for other AbstractArray, where nothing, nothing are the unknown outer bounds of the lookup. They are not needed for Points as the outer values are the outer bounds. But they can be specified manually for Intervals

    • Empty dimensions or dimension types are assigned NoLookup() ranges that can't be used with selectors as they hold no values.

DimSelector ​

We can also index with arrays of selectors DimSelectors. These are like CartesianIndices or DimIndices, but holding the Selectors At, Near, or Contains.

julia
julia> A = rand(X(1.0:0.2:2.0), Y(10:2:20))
╭─────────────────────────╮
│ 6×6 DimArray{Float64,2} │
├─────────────────────────┴────────────────────────────────────────────── dims ┐
  ↓ X Sampled{Float64} 1.0:0.2:2.0 ForwardOrdered Regular Points,
  → Y Sampled{Int64} 10:2:20 ForwardOrdered Regular Points
└──────────────────────────────────────────────────────────────────────────────┘
 ↓ →  10          12          14          16          18         20
 1.0   0.0775482   0.263531    0.361536    0.508614    0.89559    0.564657
 1.2   0.838572    0.388768    0.44818     0.0300922   0.595409   0.952489
 1.4   0.166221    0.0689895   0.499307    0.228081    0.473944   0.747695
 1.6   0.11787     0.143478    0.0831822   0.528009    0.631878   0.993001
 1.8   0.0905873   0.909826    0.557418    0.229792    0.29205    0.699118
 2.0   0.495624    0.810377    0.120845    0.574678    0.371583   0.460775

We can define another array with partly matching indices

julia
julia> B = rand(X(1.0:0.04:2.0), Y(20:-1:10))
╭───────────────────────────╮
│ 26×11 DimArray{Float64,2} │
├───────────────────────────┴──────────────────────────────────────────── dims ┐
  ↓ X Sampled{Float64} 1.0:0.04:2.0 ForwardOrdered Regular Points,
  → Y Sampled{Int64} 20:-1:10 ReverseOrdered Regular Points
└──────────────────────────────────────────────────────────────────────────────┘
 ↓ →   20         19           …  12          11          10
 1.0    0.751498   0.988013        0.966133    0.465112    0.227865
 1.04   0.241791   0.34589         0.0773793   0.728852    0.462859
 1.08   0.412631   0.0143204       0.454134    0.0417586   0.286
 1.12   0.300304   0.660657        0.712342    0.806495    0.727464
 ⋮                             ⋱               ⋮
 1.84   0.752331   0.491911        0.221313    0.72454     0.00869054
 1.88   0.296393   0.00428189      0.951789    0.142406    0.331422
 1.92   0.930699   0.777408        0.0661958   0.0737984   0.539197
 1.96   0.744107   0.360032    …   0.0699673   0.264001    0.27206
 2.0    0.635373   0.533041        0.574526    0.531012    0.503183

And we can simply select values from B with selectors from A:

julia
julia> B[DimSelectors(A)]
╭─────────────────────────╮
│ 6×6 DimArray{Float64,2} │
├─────────────────────────┴────────────────────────────────────────────── dims ┐
  ↓ X Sampled{Float64} [1.0, 1.2, …, 1.8, 2.0] ForwardOrdered Irregular Points,
  → Y Sampled{Int64} [10, 12, …, 18, 20] ReverseOrdered Irregular Points
└──────────────────────────────────────────────────────────────────────────────┘
 ↓ →  10          12         14          16         18         20
 1.0   0.227865    0.966133   0.40169     0.990114   0.358535   0.751498
 1.2   0.235356    0.675459   0.426312    0.768719   0.569347   0.399781
 1.4   0.508938    0.166186   0.0811277   0.342499   0.819291   0.998484
 1.6   0.573391    0.740107   0.817199    0.690159   0.527254   0.634257
 1.8   0.0255364   0.895115   0.519705    0.626382   0.171961   0.6063
 2.0   0.503183    0.574526   0.98981     0.450629   0.545522   0.635373

If the lookups aren't aligned, we can use Near instead of At, which is like doing a nearest neighbor interpolation:

julia
julia> C = rand(X(1.0:0.007:2.0), Y(10.0:0.9:30))
╭────────────────────────────╮
│ 143×23 DimArray{Float64,2} │
├────────────────────────────┴─────────────────────────────────────────── dims ┐
  ↓ X Sampled{Float64} 1.0:0.007:1.994 ForwardOrdered Regular Points,
  → Y Sampled{Float64} 10.0:0.9:29.8 ForwardOrdered Regular Points
└──────────────────────────────────────────────────────────────────────────────┘
 ↓ →    10.0       10.9        …  28.0         28.9        29.8
 1.0     0.341785   0.946655       0.954153     0.0821229   0.828315
 1.007   0.275784   0.22401        0.199364     0.485952    0.193179
 1.014   0.659836   0.832345       0.672556     0.0760101   0.950007
 1.021   0.168617   0.890377       0.258764     0.195897    0.747615
 ⋮                             ⋱
 1.966   0.426134   0.713457       0.693866     0.899783    0.166822
 1.973   0.601787   0.904115   …   0.0670244    0.42964     0.639039
 1.98    0.902689   0.231483       0.18906      0.670627    0.67989
 1.987   0.454047   0.92977        0.493784     0.998661    0.276742
 1.994   0.846736   0.0923954      0.00258281   0.273104    0.562491
julia
julia> C[DimSelectors(A; selectors=Near)]
╭─────────────────────────╮
│ 6×6 DimArray{Float64,2} │
├─────────────────────────┴────────────────────────────────────────────── dims ┐
  ↓ X Sampled{Float64} [1.0, 1.203, …, 1.798, 1.994] ForwardOrdered Irregular Points,
  → Y Sampled{Float64} [10.0, 11.8, …, 18.1, 19.9] ForwardOrdered Irregular Points
└──────────────────────────────────────────────────────────────────────────────┘
 ↓ →    10.0       11.8        13.6        16.3       18.1        19.9
 1.0     0.341785   0.0506805   0.621422    0.775495   0.198058    0.168554
 1.203   0.333095   0.953651    0.943783    0.789816   0.215448    0.106973
 1.399   0.641266   0.694096    0.0724261   0.603258   0.994395    0.570396
 1.602   0.827349   0.662227    0.682734    0.432028   0.707314    0.0542529
 1.798   0.192274   0.243368    0.53407     0.513909   0.0154473   0.450775
 1.994   0.846736   0.573469    0.0410007   0.513088   0.221908    0.0135194