API Reference

Arrays

# DimensionalData.AbstractBasicDimArray — Type.

AbstractBasicDimArray <: AbstractArray

The abstract supertype for all arrays with a dims method that returns a Tuple of Dimension

Only keyword rebuild is guaranteed to work with AbstractBasicDimArray.

source

# DimensionalData.AbstractDimArray — Type.

AbstractDimArray <: AbstractBasicArray

Abstract supertype for all "dim" arrays.

These arrays return a Tuple of Dimension from a dims method, and can be rebuilt using rebuild.

parent must return the source array.

They should have metadata, name and refdims methods, although these are optional.

A rebuild method for AbstractDimArray must accept data, dims, refdims, name, metadata arguments.

Indexing AbstractDimArray with non-range AbstractArray has undefined effects on the Dimension index. Use forward-ordered arrays only"

source

# DimensionalData.DimArray — Type.

DimArray <: AbstractDimArray

DimArray(data, dims, refdims, name, metadata)
DimArray(data, dims::Tuple; refdims=(), name=NoName(), metadata=NoMetadata())

The main concrete subtype of AbstractDimArray.

DimArray maintains and updates its Dimensions through transformations and moves dimensions to reference dimension refdims after reducing operations (like e.g. mean).

Arguments

• data: An AbstractArray.

• dims: A Tuple of Dimension

• name: A string name for the array. Shows in plots and tables.

• refdims: refence dimensions. Usually set programmatically to track past slices and reductions of dimension for labelling and reconstruction.

• metadata: Dict or Metadata object, or NoMetadata()

Indexing can be done with all regular indices, or with Dimensions and/or Selectors.

Indexing AbstractDimArray with non-range AbstractArray has undefined effects on the Dimension index. Use forward-ordered arrays only"

Example:

using Dates, DimensionalData

ti = (Ti(DateTime(2001):Month(1):DateTime(2001,12)),
x = X(10:10:100))
A = DimArray(rand(12,10), (ti, x), "example")

julia> A[X(Near([12, 35])), Ti(At(DateTime(2001,5)))];

julia> A[Near(DateTime(2001, 5, 4)), Between(20, 50)];

source

Shorthand AbstractDimArray constructors:

# Base.fill — Function.

Base.fill(x, dims::Dimension...; kw...) => DimArray
Base.fill(x, dims::Tuple{Vararg{Dimension}}; kw...) => DimArray

Create a DimArray with a fill value of x.

There are two kinds of Dimension value acepted:

• A Dimension holding an AbstractVector will set the dimension index to that AbstractVector, and detect the dimension lookup.

• A Dimension holding an Integer will set the length of the axis, and set the dimension lookup to NoLookup.

Keywords are the same as for DimArray.

Example

julia> using DimensionalData, Random; Random.seed!(123)
julia> rand(Bool, X(2), Y(4))
╭──────────────────────╮
│ 2×4 DimArray{Bool,2} │
├──────────────── dims ┤
  ↓ X, → Y
└──────────────────────┘
 0  0  0  0
 1  0  0  1

source

# Base.rand — Function.

Base.rand(x, dims::Dimension...; kw...) => DimArray
Base.rand(x, dims::Tuple{Vararg{Dimension}}; kw...) => DimArray
Base.rand(r::AbstractRNG, x, dims::Tuple{Vararg{Dimension}}; kw...) => DimArray
Base.rand(r::AbstractRNG, x, dims::Dimension...; kw...) => DimArray

Create a DimArray of random values.

There are two kinds of Dimension value acepted:

• A Dimension holding an AbstractVector will set the dimension index to that AbstractVector, and detect the dimension lookup.

• A Dimension holding an Integer will set the length of the axis, and set the dimension lookup to NoLookup.

Keywords are the same as for DimArray.

Example

julia> using DimensionalData

julia> rand(Bool, X(2), Y(4))
╭──────────────────────╮
│ 2×4 DimArray{Bool,2} │
├──────────────── dims ┤
  ↓ X, → Y
└──────────────────────┘
 1  1  0  0
 0  1  1  0

julia> rand(X([:a, :b, :c]), Y(100.0:50:200.0))
╭─────────────────────────╮
│ 3×3 DimArray{Float64,2} │
├─────────────────────────┴───────────────────────────────────── dims ┐
  ↓ X Categorical{Symbol} [:a, :b, :c] ForwardOrdered,
  → Y Sampled{Float64} 100.0:50.0:200.0 ForwardOrdered Regular Points
└─────────────────────────────────────────────────────────────────────┘
 ↓ →  100.0       150.0       200.0
  :a    0.624539    0.559166    0.813246
  :b    0.947442    0.664213    0.284669
  :c    0.695604    0.564835    0.156286

source

# Base.zeros — Function.

Base.zeros(x, dims::Dimension...; kw...) => DimArray
Base.zeros(x, dims::Tuple{Vararg{Dimension}}; kw...) => DimArray

Create a DimArray of zeros.

There are two kinds of Dimension value acepted:

• A Dimension holding an AbstractVector will set the dimension index to that AbstractVector, and detect the dimension lookup.

• A Dimension holding an Integer will set the length of the axis, and set the dimension lookup to NoLookup.

Keywords are the same as for DimArray.

Example

julia> using DimensionalData
julia> zeros(Bool, X(2), Y(4))
╭──────────────────────╮
│ 2×4 DimArray{Bool,2} │
├──────────────── dims ┤
  ↓ X, → Y
└──────────────────────┘
 0  0  0  0
 0  0  0  0

julia> zeros(X([:a, :b, :c]), Y(100.0:50:200.0))
╭─────────────────────────╮
│ 3×3 DimArray{Float64,2} │
├─────────────────────────┴───────────────────────────────────── dims ┐
  ↓ X Categorical{Symbol} [:a, :b, :c] ForwardOrdered,
  → Y Sampled{Float64} 100.0:50.0:200.0 ForwardOrdered Regular Points
└─────────────────────────────────────────────────────────────────────┘
 ↓ →  100.0  150.0  200.0
  :a    0.0    0.0    0.0
  :b    0.0    0.0    0.0
  :c    0.0    0.0    0.0

source

# Base.ones — Function.

Base.ones(x, dims::Dimension...; kw...) => DimArray
Base.ones(x, dims::Tuple{Vararg{Dimension}}; kw...) => DimArray

Create a DimArray of ones.

There are two kinds of Dimension value acepted:

• A Dimension holding an AbstractVector will set the dimension index to that AbstractVector, and detect the dimension lookup.

• A Dimension holding an Integer will set the length of the axis, and set the dimension lookup to NoLookup.

Keywords are the same as for DimArray.

Example

julia> using DimensionalData
julia> ones(Bool, X(2), Y(4))
╭──────────────────────╮
│ 2×4 DimArray{Bool,2} │
├──────────────── dims ┤
  ↓ X, → Y
└──────────────────────┘
 1  1  1  1
 1  1  1  1

julia> ones(X([:a, :b, :c]), Y(100.0:50:200.0))
╭─────────────────────────╮
│ 3×3 DimArray{Float64,2} │
├─────────────────────────┴───────────────────────────────────── dims ┐
  ↓ X Categorical{Symbol} [:a, :b, :c] ForwardOrdered,
  → Y Sampled{Float64} 100.0:50.0:200.0 ForwardOrdered Regular Points
└─────────────────────────────────────────────────────────────────────┘
 ↓ →  100.0  150.0  200.0
  :a    1.0    1.0    1.0
  :b    1.0    1.0    1.0
  :c    1.0    1.0    1.0

source

Functions for getting information from objects:

# DimensionalData.Dimensions.dims — Function.

dims(x, [dims::Tuple]) => Tuple{Vararg{Dimension}}
dims(x, dim) => Dimension

Return a tuple of Dimensions for an object, in the order that matches the axes or columns of the underlying data.

dims can be Dimension, Dimension types, or Symbols for Dim{Symbol}.

The default is to return nothing.

source

dims(x, query) => Tuple{Vararg{Dimension}}
dims(x, query...) => Tuple{Vararg{Dimension}}

Get the dimension(s) matching the type(s) of the query dimension.

Lookup can be an Int or an Dimension, or a tuple containing any combination of either.

Arguments

• x: any object with a dims method, or a Tuple of Dimension.

• query: Tuple or a single Dimension or Dimension Type.

Example

julia> using DimensionalData

julia> A = DimArray(ones(2, 3, 2), (X, Y, Z))
╭───────────────────────────╮
│ 2×3×2 DimArray{Float64,3} │
├───────────────────── dims ┤
  ↓ X, → Y, ↗ Z
└───────────────────────────┘
[:, :, 1]
 1.0  1.0  1.0
 1.0  1.0  1.0

julia> dims(A, (X, Y))
↓ X, → Y

source

# DimensionalData.Dimensions.refdims — Function.

refdims(x, [dims::Tuple]) => Tuple{Vararg{Dimension}}
refdims(x, dim) => Dimension

Reference dimensions for an array that is a slice or view of another array with more dimensions.

slicedims(a, dims) returns a tuple containing the current new dimensions and the new reference dimensions. Refdims can be stored in a field or disgarded, as it is mostly to give context to plots. Ignoring refdims will simply leave some captions empty.

The default is to return an empty Tuple ().

source

# DimensionalData.Dimensions.Lookups.metadata — Function.

metadata(x) => (object metadata)
metadata(x, dims::Tuple)  => Tuple (Dimension metadata)
metadata(xs::Tuple) => Tuple

Returns the metadata for an object or for the specified dimension(s)

Second argument dims can be Dimensions, Dimension types, or Symbols for Dim{Symbol}.

source

# DimensionalData.Dimensions.name — Function.

name(x) => Symbol
name(xs:Tuple) => NTuple{N,Symbol}
name(x, dims::Tuple) => NTuple{N,Symbol}
name(x, dim) => Symbol

Get the name of an array or Dimension, or a tuple of of either as a Symbol.

Second argument dims can be Dimensions, Dimension types, or Symbols for Dim{Symbol}.

source

# DimensionalData.Dimensions.otherdims — Function.

otherdims(x, query) => Tuple{Vararg{Dimension,N}}

Get the dimensions of an object not in query.

Arguments

• x: any object with a dims method, a Tuple of Dimension.

• query: Tuple or single Dimension or dimension Type.

• f: <: by default, but can be >: to match abstract types to concrete types.

A tuple holding the unmatched dimensions is always returned.

Example

julia> using DimensionalData, DimensionalData.Dimensions

julia> A = DimArray(ones(10, 10, 10), (X, Y, Z));

julia> otherdims(A, X)
↓ Y, → Z

julia> otherdims(A, (Y, Z))
↓ X

source

# DimensionalData.Dimensions.dimnum — Function.

dimnum(x, query::Tuple) => NTuple{Int}
dimnum(x, query) => Int

Get the number(s) of Dimension(s) as ordered in the dimensions of an object.

Arguments

• x: any object with a dims method, a Tuple of Dimension or a single Dimension.

• query: Tuple, Array or single Dimension or dimension Type.

The return type will be a Tuple of Int or a single Int, depending on wether query is a Tuple or single Dimension.

Example

julia> using DimensionalData

julia> A = DimArray(ones(10, 10, 10), (X, Y, Z));

julia> dimnum(A, (Z, X, Y))
(3, 1, 2)

julia> dimnum(A, Y)
2

source

# DimensionalData.Dimensions.hasdim — Function.

hasdim([f], x, query::Tuple) => NTUple{Bool}
hasdim([f], x, query...) => NTUple{Bool}
hasdim([f], x, query) => Bool

Check if an object x has dimensions that match or inherit from the query dimensions.

Arguments

• x: any object with a dims method, a Tuple of Dimension or a single Dimension.

• query: Tuple or single Dimension or dimension Type.

• f: <: by default, but can be >: to match abstract types to concrete types.

Check if an object or tuple contains an Dimension, or a tuple of dimensions.

Example

julia> using DimensionalData

julia> A = DimArray(ones(10, 10, 10), (X, Y, Z));

julia> hasdim(A, X)
true

julia> hasdim(A, (Z, X, Y))
(true, true, true)

julia> hasdim(A, Ti)
false

source

Multi-array datasets

# DimensionalData.AbstractDimStack — Type.

AbstractDimStack

Abstract supertype for dimensional stacks.

These have multiple layers of data, but share dimensions.

Notably, their behaviour lies somewhere between a DimArray and a NamedTuple:

• indexing with a Symbol as in dimstack[:symbol] returns a DimArray layer.

• iteration and map apply over array layers, as indexed with a Symbol.

• getindex and many base methods are applied as for DimArray - to avoid the need to allways use map.

This design gives very succinct code when working with many-layered, mixed-dimension objects. But it may be jarring initially - the most surprising outcome is that dimstack[1] will return a NamedTuple of values for the first index in all layers, while first(dimstack) will return the first value of the iterator - the DimArray for the first layer.

See DimStack for the concrete implementation. Most methods are defined on the abstract type.

To extend AbstractDimStack, implement argument and keyword version of rebuild and also rebuild_from_arrays.

The constructor of an AbstractDimStack must accept a NamedTuple.

source

# DimensionalData.DimStack — Type.

DimStack <: AbstractDimStack

DimStack(data::AbstractDimArray...; kw...)
DimStack(data::Tuple{Vararg{AbstractDimArray}}; kw...)
DimStack(data::NamedTuple{Keys,Vararg{AbstractDimArray}}; kw...)
DimStack(data::NamedTuple, dims::DimTuple; metadata=NoMetadata(); kw...)

DimStack holds multiple objects sharing some dimensions, in a NamedTuple.

Notably, their behaviour lies somewhere between a DimArray and a NamedTuple:

• indexing with a Symbol as in dimstack[:symbol] returns a DimArray layer.

• iteration and map apply over array layers, as indexed with a Symbol.

• getindex or view with Int, Dimensions or Selectors that resolve to Int will return a NamedTuple of values from each layer in the stack. This has very good performace, and avoids the need to always use map.

• getindex or view with a Vector or Colon will return another DimStack where all data layers have been sliced.

• setindex! must pass a Tuple or NamedTuple matching the layers.

• many base and Statistics methods (sum, mean etc) will work as for a DimArray again removing the need to use map.

function DimStack(A::AbstractDimArray; layersfrom=nothing, name=nothing, metadata=metadata(A), refdims=refdims(A), kw... )

For example, here we take the mean over the time dimension for all layers :

mean(mydimstack; dims=Ti)

And this equivalent to:

map(A -> mean(A; dims=Ti), mydimstack)

This design gives succinct code when working with many-layered, mixed-dimension objects.

But it may be jarring initially - the most surprising outcome is that dimstack[1] will return a NamedTuple of values for the first index in all layers, while first(dimstack) will return the first value of the iterator - the DimArray for the first layer.

DimStack can be constructed from multiple AbstractDimArray or a NamedTuple of AbstractArray and a matching dims tuple.

Most Base and Statistics methods that apply to AbstractArray can be used on all layers of the stack simulataneously. The result is a DimStack, or a NamedTuple if methods like mean are used without dims arguments, and return a single non-array value.

Example

julia> using DimensionalData

julia> A = [1.0 2.0 3.0; 4.0 5.0 6.0];

julia> dimz = (X([:a, :b]), Y(10.0:10.0:30.0))
↓ X [:a, :b],
→ Y 10.0:10.0:30.0

julia> da1 = DimArray(1A, dimz; name=:one);


julia> da2 = DimArray(2A, dimz; name=:two);


julia> da3 = DimArray(3A, dimz; name=:three);


julia> s = DimStack(da1, da2, da3);


julia> s[At(:b), At(10.0)]
(one = 4.0, two = 8.0, three = 12.0)

julia> s[X(At(:a))] isa DimStack
true

source

Dimension generators

# DimensionalData.DimIndices — Type.

DimIndices <: AbstractArray

DimIndices(x)
DimIndices(dims::Tuple)
DimIndices(dims::Dimension)

Like CartesianIndices, but for Dimensions. Behaves as an Array of Tuple of Dimension(i) for all combinations of the axis indices of dims.

This can be used to view/index into arbitrary dimensions over an array, and is especially useful when combined with otherdims, to iterate over the indices of unknown dimension.

DimIndices can be used directly in getindex like CartesianIndices, and freely mixed with individual Dimensions or tuples of Dimension.

Example

Index a DimArray with DimIndices.

Notice that unlike CartesianIndices, it doesn't matter if the dimensions are not in the same order. Or even if they are not all contained in each.

julia> di = DimIndices((X(1:2:4), Y(1:2:4))) ╭──────────────────────────────────────────────╮ │ 2×2 DimIndices{Tuple{X{Int64}, Y{Int64}},2}  │ ├──────────────────────────────────────────────┴── dims ┐   ↓ X 1:2:3,   → Y 1:2:3 └───────────────────────────────────────────────────────┘  ↓ X 1, → Y 1  ↓ X 1, → Y 3  ↓ X 3, → Y 1  ↓ X 3, → Y 3

julia> A[di] # Index A with these indices dims(d) = (X{StepRange{Int64, Int64}}(1:2:3), Y{StepRange{Int64, Int64}}(1:2:3)) ╭─────────────────────────╮ │ 2×2 DimArray{Float64,2} │ ├─────────────────────────┴─────────────────────────────────── dims ┐   ↓ Y Sampled{Float64} 0.0:0.6:0.6 ForwardOrdered Regular Points,   → X Categorical{Char} 'a':2:'c' ForwardOrdered └───────────────────────────────────────────────────────────────────┘  ↓ →   'a'       'c'  0.0  0.513225  0.771862  0.6  0.837621  0.441426


[source](https://github.com/rafaqz/DimensionalData.jl/blob/d8a3d4ba312c264743e2fca80684f7d552949649/src/dimindices.jl#L48-L108)

</div>
<br>
<div style='border-width:1px; border-style:solid; border-color:black; padding: 1em; border-radius: 25px;'>
<a id='DimensionalData.DimSelectors' href='#DimensionalData.DimSelectors'>#</a>&nbsp;<b><u>DimensionalData.DimSelectors</u></b> &mdash; <i>Type</i>.




```julia
DimSelectors <: AbstractArray

DimSelectors(x; selectors, atol...)
DimSelectors(dims::Tuple; selectors, atol...)
DimSelectors(dims::Dimension; selectors, atol...)

Like DimIndices, but returns Dimensions holding the chosen Selectors.

Indexing into another AbstractDimArray with DimSelectors is similar to doing an interpolation.

Keywords

• selectors: Near, At or Contains, or a mixed tuple of these. At is the default, meaning only exact or within atol values are used.

• atol: used for At selectors only, as the atol value.

Example

Here we can interpolate a DimArray to the lookups of another DimArray using DimSelectors with Near. This is essentially equivalent to nearest neighbour interpolation.

julia> A = rand(X(1.0:3.0:30.0), Y(1.0:5.0:30.0), Ti(1:2));

julia> target = rand(X(1.0:10.0:30.0), Y(1.0:10.0:30.0));

julia> A[DimSelectors(target; selectors=Near), Ti=2]
╭───────────────────────────╮
│ 3×3×2 DimArray{Float64,3} │
├───────────────────────────┴──────────────────────────────────────── dims ┐
  ↓ X  Sampled{Float64} [1.0, 10.0, 22.0] ForwardOrdered Irregular Points,
  → Y  Sampled{Float64} [1.0, 11.0, 21.0] ForwardOrdered Irregular Points,
└──────────────────────────────────────────────────────────────────────────┘
  ↓ →  1.0       11.0       21.0
  1.0  0.473548   0.773863   0.541381
 10.0  0.951457   0.176647   0.968292
 22.0  0.822979   0.980585   0.544853

Using At would make sure we only use exact interpolation, while Contains with sampleing of Intervals would make sure that each values is taken only from an Interval that is present in the lookups.

source

# DimensionalData.DimPoints — Type.

DimPoints <: AbstractArray

DimPoints(x; order)
DimPoints(dims::Tuple; order)
DimPoints(dims::Dimension; order)

Like CartesianIndices, but for the point values of the dimension index. Behaves as an Array of Tuple lookup values (whatever they are) for all combinations of the lookup values of dims.

Either a Dimension, a Tuple of Dimension or an object x that defines a dims method can be passed in.

Keywords

• order: determines the order of the points, the same as the order of dims by default.

source

Tables.jl/TableTraits.jl interface

# DimensionalData.AbstractDimTable — Type.

AbstractDimTable <: Tables.AbstractColumns

Abstract supertype for dim tables

source

# DimensionalData.DimTable — Type.

DimTable <: AbstractDimTable

DimTable(s::AbstractDimStack; mergedims=nothing)
DimTable(x::AbstractDimArray; layersfrom=nothing, mergedims=nothing)
DimTable(xs::Vararg{AbstractDimArray}; layernames=nothing, mergedims=nothing)

Construct a Tables.jl/TableTraits.jl compatible object out of an AbstractDimArray or AbstractDimStack.

This table will have columns for the array data and columns for each Dimension index, as a [DimColumn]. These are lazy, and generated as required.

Column names are converted from the dimension types using DimensionalData.name. This means type Ti becomes the column name :Ti, and Dim{:custom} becomes :custom.

To get dimension columns, you can index with Dimension (X()) or Dimension type (X) as well as the regular Int or Symbol.

Keywords

• mergedims: Combine two or more dimensions into a new dimension.

• layersfrom: Treat a dimension of an AbstractDimArray as layers of an AbstractDimStack.

Example

julia> using DimensionalData, Tables

julia> a = DimArray(ones(16, 16, 3), (X, Y, Dim{:band}))
╭─────────────────────────────╮
│ 16×16×3 DimArray{Float64,3} │
├─────────────────────── dims ┤
  ↓ X, → Y, ↗ band
└─────────────────────────────┘
[:, :, 1]
 1.0  1.0  1.0  1.0  1.0  1.0  1.0  1.0  …  1.0  1.0  1.0  1.0  1.0  1.0  1.0
 1.0  1.0  1.0  1.0  1.0  1.0  1.0  1.0     1.0  1.0  1.0  1.0  1.0  1.0  1.0
 1.0  1.0  1.0  1.0  1.0  1.0  1.0  1.0     1.0  1.0  1.0  1.0  1.0  1.0  1.0
 1.0  1.0  1.0  1.0  1.0  1.0  1.0  1.0     1.0  1.0  1.0  1.0  1.0  1.0  1.0
 1.0  1.0  1.0  1.0  1.0  1.0  1.0  1.0     1.0  1.0  1.0  1.0  1.0  1.0  1.0
 1.0  1.0  1.0  1.0  1.0  1.0  1.0  1.0  …  1.0  1.0  1.0  1.0  1.0  1.0  1.0
 1.0  1.0  1.0  1.0  1.0  1.0  1.0  1.0     1.0  1.0  1.0  1.0  1.0  1.0  1.0
 1.0  1.0  1.0  1.0  1.0  1.0  1.0  1.0     1.0  1.0  1.0  1.0  1.0  1.0  1.0
 1.0  1.0  1.0  1.0  1.0  1.0  1.0  1.0     1.0  1.0  1.0  1.0  1.0  1.0  1.0
 1.0  1.0  1.0  1.0  1.0  1.0  1.0  1.0     1.0  1.0  1.0  1.0  1.0  1.0  1.0
 1.0  1.0  1.0  1.0  1.0  1.0  1.0  1.0  …  1.0  1.0  1.0  1.0  1.0  1.0  1.0
 1.0  1.0  1.0  1.0  1.0  1.0  1.0  1.0     1.0  1.0  1.0  1.0  1.0  1.0  1.0
 1.0  1.0  1.0  1.0  1.0  1.0  1.0  1.0     1.0  1.0  1.0  1.0  1.0  1.0  1.0
 1.0  1.0  1.0  1.0  1.0  1.0  1.0  1.0     1.0  1.0  1.0  1.0  1.0  1.0  1.0
 1.0  1.0  1.0  1.0  1.0  1.0  1.0  1.0     1.0  1.0  1.0  1.0  1.0  1.0  1.0
 1.0  1.0  1.0  1.0  1.0  1.0  1.0  1.0  …  1.0  1.0  1.0  1.0  1.0  1.0  1.0

julia>

source

Group by methods

For transforming DimensionalData objects:

# DataAPI.groupby — Function.

groupby(A::Union{AbstractDimArray,AbstractDimStack}, dims::Pair...)
groupby(A::Union{AbstractDimArray,AbstractDimStack}, dims::Dimension{<:Callable}...)

Group A by grouping functions or Bins over multiple dimensions.

Arguments

• A: any AbstractDimArray or AbsractDimStack.

• dims: Pairs such as groups = groupby(A, :dimname => groupingfunction) or wrapped Dimensions like groups = groupby(A, DimType(groupingfunction)). Instead of a grouping function Bins can be used to specify group bins.

Return value

A DimGroupByArray is returned, which is basically a regular AbstractDimArray but holding the grouped AbstractDimArray or AbstractDimStrack. Its dims hold the sorted values returned by the grouping function/s.

Base julia and package methods work on DimGroupByArray as for any other AbstractArray of AbstractArray.

It is common to broadcast or map a reducing function over groups, such as mean or sum, like mean.(groups) or map(mean, groups). This will return a regular DimArray, or DimGroupByArray if dims keyword is used in the reducing function or it otherwise returns an AbstractDimArray or AbstractDimStack.

Example

Group some data along the time dimension:

julia> using DimensionalData, Dates

julia> A = rand(X(1:0.1:20), Y(1:20), Ti(DateTime(2000):Day(3):DateTime(2003)));

julia> groups = groupby(A, Ti => month) # Group by month
╭────────────────────────────────────────╮
│ 12-element DimGroupByArray{DimArray,1} │
├────────────────────────────────────────┴──────────────────────── dims ┐
  ↓ Ti Sampled{Int64} [1, 2, …, 11, 12] ForwardOrdered Irregular Points
├───────────────────────────────────────────────────────────── metadata ┤
  Dict{Symbol, Any} with 1 entry:
  :groupby => (Ti{typeof(month)}(month),)
├─────────────────────────────────────────────────────────── group dims ┤
  ↓ X, → Y, ↗ Ti
└───────────────────────────────────────────────────────────────────────┘
  1  191×20×32 DimArray
  2  191×20×28 DimArray
  3  191×20×31 DimArray
  4  191×20×30 DimArray
  ⋮
  9  191×20×30 DimArray
 10  191×20×31 DimArray
 11  191×20×30 DimArray
 12  191×20×31 DimArray

And take the mean:

julia> groupmeans = mean.(groups) # Take the monthly mean
╭────────────────────────────────╮
│ 12-element DimArray{Float64,1} │
├────────────────────────────────┴──────────────────────────────── dims ┐
  ↓ Ti Sampled{Int64} [1, 2, …, 11, 12] ForwardOrdered Irregular Points
├───────────────────────────────────────────────────────────── metadata ┤
  Dict{Symbol, Any} with 1 entry:
  :groupby => (Ti{typeof(month)}(month),)
└───────────────────────────────────────────────────────────────────────┘
  1  0.499943
  2  0.499352
  3  0.499289
  4  0.499899
  ⋮
 10  0.500755
 11  0.498912
 12  0.500352

Calculate daily anomalies from the monthly mean. Notice we map a broadcast .- rather than -. This is because the size of the arrays to not match after application of mean.

julia> map(.-, groupby(A, Ti=>month), mean.(groupby(A, Ti=>month), dims=Ti));

Or do something else with Y:

julia> groupmeans = mean.(groupby(A, Ti=>month, Y=>isodd))
╭──────────────────────────╮
│ 12×2 DimArray{Float64,2} │
├──────────────────────────┴─────────────────────────────────────── dims ┐
  ↓ Ti Sampled{Int64} [1, 2, …, 11, 12] ForwardOrdered Irregular Points,
  → Y  Sampled{Bool} [false, true] ForwardOrdered Irregular Points
├──────────────────────────────────────────────────────────────────────── metadata ┐
  Dict{Symbol, Any} with 1 entry:
  :groupby => (Ti{typeof(month)}(month), Y{typeof(isodd)}(isodd))
└──────────────────────────────────────────────────────────────────────────────────┘
  ↓ →  false         true
  1        0.500465     0.499421
  2        0.498681     0.500024
  ⋮
 10        0.500183     0.501327
 11        0.497746     0.500079
 12        0.500287     0.500417

source

# DimensionalData.DimGroupByArray — Type.

DimGroupByArray <: AbstractDimArray

DimGroupByArray is essentially a DimArray but holding the results of a groupby operation.

Its dimensions are the sorted results of the grouping functions used in groupby.

This wrapper allows for specialisations on later broadcast or reducing operations, e.g. for chunk reading with DiskArrays.jl, because we know the data originates from a single array.

source

# DimensionalData.Bins — Type.

Bins(f, bins; labels, pad)
Bins(bins; labels, pad)

Specify bins to reduce groups after applying function f.

• f a grouping function of the lookup values, by default identity.

• bins:

  • an Integer will divide the group values into equally spaced sections.

  • an AbstractArray of values will be treated as exact matches for the return value of f. For example, 1:3 will create 3 bins - 1, 2, 3.

  • an AbstractArray of IntervalSets.Interval can be used to explictly define the intervals. Overlapping intervals have undefined behaviour.

Keywords

• pad: fraction of the total interval to pad at each end when Bins contains an Integer. This avoids losing the edge values. Note this is a messy solution - it will often be prefereble to manually specify a Vector of chosen Intervals rather than relying on passing an Integer and pad.

• labels: a list of descriptive labels for the bins. The labels need to have the same length as bins.

When the return value of f is a tuple, binning is applied to the last value of the tuples.

source

# DimensionalData.ranges — Function.

ranges(A::AbsttactRange{<:Integer})

Generate a Vector of UnitRange with length step(A)

source

# DimensionalData.intervals — Function.

intervals(A::AbstractRange)

Generate a Vector of UnitRange with length step(A)

source

# DimensionalData.CyclicBins — Type.

CyclicBins(f; cycle, start, step, labels)

Cyclic bins to reduce groups after applying function f. Groups can wrap around the cycle. This is used for grouping in seasons, months and hours but can also be used for custom cycles.

• f a grouping function of the lookup values, by default identity.

Keywords

• cycle: the length of the cycle, in return values of f.

• start: the start of the cycle: a return value of f.

• step the number of sequential values to group.

• labels: either a vector of labels matching the number of groups, or a function that generates labels from Vector{Int} of the selected bins.

When the return value of f is a tuple, binning is applied to the last value of the tuples.

source

# DimensionalData.seasons — Function.

seasons(; [start=Dates.December, labels])

Generates CyclicBins for three month periods.

Keywords

• start: By default seasons start in December, but any integer 1:12 can be used.

• labels: either a vector of four labels, or a function that generates labels from Vector{Int} of the selected quartals.

source

# DimensionalData.months — Function.

months(step; [start=Dates.January, labels])

Generates CyclicBins for grouping to arbitrary month periods. These can wrap around the end of a year.

• step the number of months to group.

Keywords

• start: By default months start in January, but any integer 1:12 can be used.

• labels: either a vector of labels matching the number of groups, or a function that generates labels from Vector{Int} of the selected months.

source

# DimensionalData.hours — Function.

hours(step; [start=0, labels])

Generates CyclicBins for grouping to arbitrary hour periods. These can wrap around the end of the day.

• steps the number of hours to group.

Keywords

• start: By default seasons start at 0, but any integer 1:24 can be used.

• labels: either a vector of four labels, or a function that generates labels from Vector{Int} of the selected hours of the day.

source

Utility methods

For transforming DimensionalData objects:

# DimensionalData.Dimensions.Lookups.set — Function.

set(x, val)
set(x, args::Pairs...) => x with updated field/s
set(x, args...; kw...) => x with updated field/s
set(x, args::Tuple{Vararg{Dimension}}; kw...) => x with updated field/s

set(dim::Dimension, index::AbstractArray) => Dimension
set(dim::Dimension, lookup::Lookup) => Dimension
set(dim::Dimension, lookupcomponent::LookupTrait) => Dimension
set(dim::Dimension, metadata::AbstractMetadata) => Dimension

Set the properties of an object, its internal data or the traits of its dimensions and lookup index.

As DimensionalData is so strongly typed you do not need to specify what field of a Lookup to set - there is no ambiguity.

To set fields of a Lookup you need to specify the dimension. This can be done using X => val pairs, X = val keyword arguments, or X(val) wrapped arguments.

When a Dimension or Lookup is passed to set to replace the existing ones, fields that are not set will keep their original values.

Notes:

Changing a lookup index range/vector will also update the step size and order where applicable.

Setting the Order like ForwardOrdered will not reverse the array or dimension to match. Use reverse and reorder to do this.

Examples

julia> using DimensionalData; const DD = DimensionalData
DimensionalData

julia> da = DimArray(zeros(3, 4), (custom=10.0:010.0:30.0, Z=-20:010.0:10.0));

julia> set(da, ones(3, 4))
╭─────────────────────────╮
│ 3×4 DimArray{Float64,2} │
├─────────────────────────┴───────────────────────────────────────── dims ┐
  ↓ custom Sampled{Float64} 10.0:10.0:30.0 ForwardOrdered Regular Points,
  → Z      Sampled{Float64} -20.0:10.0:10.0 ForwardOrdered Regular Points
└─────────────────────────────────────────────────────────────────────────┘
  ↓ →  -20.0  -10.0  0.0  10.0
 10.0    1.0    1.0  1.0   1.0
 20.0    1.0    1.0  1.0   1.0
 30.0    1.0    1.0  1.0   1.0

Change the Dimension wrapper type:

julia> set(da, :Z => Ti, :custom => Z)
╭─────────────────────────╮
│ 3×4 DimArray{Float64,2} │
├─────────────────────────┴───────────────────────────────────── dims ┐
  ↓ Z  Sampled{Float64} 10.0:10.0:30.0 ForwardOrdered Regular Points,
  → Ti Sampled{Float64} -20.0:10.0:10.0 ForwardOrdered Regular Points
└─────────────────────────────────────────────────────────────────────┘
  ↓ →  -20.0  -10.0  0.0  10.0
 10.0    0.0    0.0  0.0   0.0
 20.0    0.0    0.0  0.0   0.0
 30.0    0.0    0.0  0.0   0.0

Change the lookup Vector:

julia> set(da, Z => [:a, :b, :c, :d], :custom => [4, 5, 6])
╭─────────────────────────╮
│ 3×4 DimArray{Float64,2} │
├─────────────────────────┴───────────────────────────────────────── dims ┐
  ↓ custom Sampled{Int64} [4, 5, 6] ForwardOrdered Regular Points,
  → Z      Sampled{Symbol} [:a, :b, :c, :d] ForwardOrdered Regular Points
└─────────────────────────────────────────────────────────────────────────┘
 ↓ →   :a   :b   :c   :d
 4    0.0  0.0  0.0  0.0
 5    0.0  0.0  0.0  0.0
 6    0.0  0.0  0.0  0.0

Change the Lookup type:

julia> set(da, Z=DD.NoLookup(), custom=DD.Sampled())
╭─────────────────────────╮
│ 3×4 DimArray{Float64,2} │
├─────────────────────────┴───────────────────────────────────────── dims ┐
  ↓ custom Sampled{Float64} 10.0:10.0:30.0 ForwardOrdered Regular Points,
  → Z     
└─────────────────────────────────────────────────────────────────────────┘
 10.0  0.0  0.0  0.0  0.0
 20.0  0.0  0.0  0.0  0.0
 30.0  0.0  0.0  0.0  0.0

Change the Sampling trait:

julia> set(da, :custom => DD.Irregular(10, 12), Z => DD.Regular(9.9))
╭─────────────────────────╮
│ 3×4 DimArray{Float64,2} │
├─────────────────────────┴─────────────────────────────────────────── dims ┐
  ↓ custom Sampled{Float64} 10.0:10.0:30.0 ForwardOrdered Irregular Points,
  → Z      Sampled{Float64} -20.0:10.0:10.0 ForwardOrdered Regular Points
└───────────────────────────────────────────────────────────────────────────┘
  ↓ →  -20.0  -10.0  0.0  10.0
 10.0    0.0    0.0  0.0   0.0
 20.0    0.0    0.0  0.0   0.0
 30.0    0.0    0.0  0.0   0.0

source

# DimensionalData.Dimensions.Lookups.rebuild — Function.

rebuild(x; kw...)

Rebuild an object struct with updated field values.

x can be a AbstractDimArray, a Dimension, Lookup or other custom types.

This is an abstraction that alows inbuilt and custom types to be rebuilt to update their fields, as most objects in DimensionalData.jl are immutable.

Rebuild is mostly automated using ConstructionBase.setproperties. It should only be defined if your object has fields with with different names to DimensionalData objects. Try not to do that!

The arguments required are defined for the abstract type that has a rebuild method.

AbstractBasicDimArray:

• dims: a Tuple of Dimension

AbstractDimArray:

• data: the parent object - an AbstractArray

• dims: a Tuple of Dimension

• refdims: a Tuple of Dimension

• name: A Symbol, or NoName and Name on GPU.

• metadata: A Dict-like object

AbstractDimStack:

• data: the parent object, often a NamedTuple

• dims, refdims, metadata

Dimension:

• val: anything.

Lookup:

• data: the parent object, an AbstractArray

• Note: argument rebuild is deprecated on AbstractDimArray and

AbstractDimStack in favour of allways using the keyword version. In future the argument version will only be used on Dimension, which only have one argument.

source

# DimensionalData.modify — Function.

modify(f, A::AbstractDimArray) => AbstractDimArray
modify(f, s::AbstractDimStack) => AbstractDimStack
modify(f, dim::Dimension) => Dimension
modify(f, x, lookupdim::Dimension) => typeof(x)

Modify the parent data, rebuilding the object wrapper without change. f must return a AbstractArray of the same size as the original.

This method is mostly useful as a way of swapping the parent array type of an object.

Example

If we have a previously-defined DimArray, we can copy it to an Nvidia GPU with:

A = DimArray(rand(100, 100), (X, Y))
modify(CuArray, A)

This also works for all the data layers in a DimStack.

source

# DimensionalData.broadcast_dims — Function.

broadcast_dims(f, sources::AbstractDimArray...) => AbstractDimArray

Broadcast function f over the AbstractDimArrays in sources, permuting and reshaping dimensions to match where required. The result will contain all the dimensions in all passed in arrays in the order in which they are found.

Arguments

• sources: AbstractDimArrays to broadcast over with f.

This is like broadcasting over every slice of A if it is sliced by the dimensions of B.

source

# DimensionalData.broadcast_dims! — Function.

broadcast_dims!(f, dest::AbstractDimArray, sources::AbstractDimArray...) => dest

Broadcast function f over the AbstractDimArrays in sources, writing to dest. sources are permuting and reshaping dimensions to match where required.

The result will contain all the dimensions in all passed in arrays, in the order in which they are found.

Arguments

• dest: AbstractDimArray to update.

• sources: AbstractDimArrays to broadcast over with f.

source

# DimensionalData.mergedims — Function.

mergedims(old_dims => new_dim) => Dimension

Return a dimension new_dim whose indices are a MergedLookup of the indices of old_dims.

source

mergedims(dims, old_dims => new_dim, others::Pair...) => dims_new

If dimensions old_dims, new_dim, etc. are found in dims, then return new dims_new where all dims in old_dims have been combined into a single dim new_dim. The returned dimension will keep only the name of new_dim. Its coords will be a MergedLookup of the coords of the dims in old_dims. New dimensions are always placed at the end of dims_new. others contains other dimension pairs to be merged.

Example

julia> using DimensionalData

julia> ds = (X(0:0.1:0.4), Y(10:10:100), Ti([0, 3, 4]))
↓ X  0.0:0.1:0.4,
→ Y  10:10:100,
↗ Ti [0, 3, 4]

julia> mergedims(ds, (X, Y) => :space)
↓ Ti    [0, 3, 4],
→ space MergedLookup{Tuple{Float64, Int64}} [(0.0, 10), (0.1, 10), …, (0.3, 100), (0.4, 100)] ↓ X, → Y

source

mergedims(A::AbstractDimArray, dim_pairs::Pair...) => AbstractDimArray
mergedims(A::AbstractDimStack, dim_pairs::Pair...) => AbstractDimStack

Return a new array or stack whose dimensions are the result of mergedims(dims(A), dim_pairs).

source

# DimensionalData.unmergedims — Function.

unmergedims(merged_dims::Tuple{Vararg{Dimension}}) => Tuple{Vararg{Dimension}}

Return the unmerged dimensions from a tuple of merged dimensions. However, the order of the original dimensions are not necessarily preserved.

source

unmergedims(A::AbstractDimArray, original_dims) => AbstractDimArray
unmergedims(A::AbstractDimStack, original_dims) => AbstractDimStack

Return a new array or stack whose dimensions are restored to their original prior to calling mergedims(A, dim_pairs).

source

# DimensionalData.reorder — Function.

reorder(A::Union{AbstractDimArray,AbstractDimStack}, order::Pair...)
reorder(A::Union{AbstractDimArray,AbstractDimStack}, order)
reorder(A::Dimension, order::Order)

Reorder every dims index/array to order, or reorder index for the the given dimension(s) in order.

order can be an Order, Dimension => Order pairs. A Tuple of Dimensions or any object that defines dims can be used in which case dimensions are

If no axis reversal is required the same objects will be returned, without allocation.

Example

using DimensionalData

# Create a DimArray
da = DimArray([1 2 3; 4 5 6], (X(10:10:20), Y(300:-100:100)))

# Reverse it
rev = reverse(da, dims=Y)

# using `da` in reorder will return it to the original order
reorder(rev, da) == da

# output
true

source

Base methods

# Base.cat — Function.

Base.cat(stacks::AbstractDimStack...; [keys=keys(stacks[1])], dims)

Concatenate all or a subset of layers for all passed in stacks.

Keywords

• keys: Tuple of Symbol for the stack keys to concatenate.

• dims: Dimension of child array to concatenate on.

Example

Concatenate the :sea_surface_temp and :humidity layers in the time dimension:

cat(stacks...; keys=(:sea_surface_temp, :humidity), dims=Ti)

source

# Base.copy! — Function.

Base.copy!(dst::AbstractArray, src::AbstractGimStack, key::Key)

Copy the stack layer key to dst, which can be any AbstractArray.

Example

Copy the :humidity layer from stack to array.

copy!(array, stack, :humidity)

source

Base.copy!(dst::AbstractDimStack, src::AbstractDimStack, [keys=keys(dst)])

Copy all or a subset of layers from one stack to another.

Example

Copy just the :sea_surface_temp and :humidity layers from src to dst.

copy!(dst::AbstractDimStack, src::AbstractDimStack, keys=(:sea_surface_temp, :humidity))

source

# Base.eachslice — Function.

Base.eachslice(stack::AbstractDimStack; dims)

Create a generator that iterates over dimensions dims of stack, returning stacks that select all the data from the other dimensions in stack using views.

The generator has size and axes equivalent to those of the provided dims.

Examples

julia> ds = DimStack((
           x=DimArray(randn(2, 3, 4), (X([:x1, :x2]), Y(1:3), Z)),
           y=DimArray(randn(2, 3, 5), (X([:x1, :x2]), Y(1:3), Ti))
       ));

julia> slices = eachslice(ds; dims=(Z, X));

julia> size(slices)
(4, 2)

julia> map(dims, axes(slices))
Z,
X Categorical{Symbol} Symbol[x1, x2] ForwardOrdered

julia> first(slices)
DimStack with dimensions:
  Y Sampled{Int64} 1:3 ForwardOrdered Regular Points,
  Ti
and 2 layers:
  :x Float64 dims: Y (3)
  :y Float64 dims: Y, Ti (3×5)

source

Most base methods work as expected, using Dimension wherever a dims keyword is used. They are not allspecifically documented here.

Name

# DimensionalData.AbstractName — Type.

AbstractName

Abstract supertype for name wrappers.

source

# DimensionalData.Name — Type.

Name <: AbstractName

Name(name::Union{Symbol,Name) => Name
Name(name::NoName) => NoName

Name wrapper. This lets arrays keep symbol names when the array wrapper neeeds to be `isbits, like for use on GPUs. It makes the name a property of the type. It's not necessary to use in normal use, a symbol is probably easier.

source

# DimensionalData.NoName — Type.

NoName <: AbstractName

NoName()

NoName specifies an array is not named, and is the default name value for all AbstractDimArrays.

source

Internal interface

# DimensionalData.DimArrayInterface — Type.

    DimArrayInterface

An Interfaces.jl Interface with mandatory components (:dims, :refdims_base, :ndims, :size, :rebuild_parent, :rebuild_dims, :rebuild_parent_kw, :rebuild_dims_kw, :rebuild) and optional components (:refdims, :name, :metadata).

This is an early stage of inteface definition, many things are not yet tested.

Pass constructed AbstractDimArrays as test data.

They must not be zero dimensional, and should test at least 1, 2, and 3 dimensions.

source

# DimensionalData.DimStackInterface — Type.

    DimStackInterface

An Interfaces.jl Interface with mandatory components (:dims, :refdims_base, :ndims, :size, :rebuild_parent, :rebuild_dims, :rebuild_layerdims, :rebuild_dims_kw, :rebuild_parent_kw, :rebuild_layerdims_kw, :rebuild) and optional components (:refdims, :metadata).

This is an early stage of inteface definition, many things are not yet tested.

Pass constructed AbstractDimArrays as test data.

They must not be zero dimensional, and should test at least 1, 2, and 3 dimensions.

source

# DimensionalData.rebuild_from_arrays — Function.

rebuild_from_arrays(s::AbstractDimStack, das::NamedTuple{<:Any,<:Tuple{Vararg{AbstractDimArray}}}; kw...)

Rebuild an AbstractDimStack from a Tuple or NamedTuple of AbstractDimArray and an existing stack.

Keywords

Keywords are simply the fields of the stack object:

• data

• dims

• refdims

• metadata

• layerdims

• layermetadata

source

# DimensionalData.show_main — Function.

show_main(io::IO, mime, A::AbstractDimArray)
show_main(io::IO, mime, A::AbstractDimStack)

Interface methods for adding the main part of show

At the least, you likely want to call:

'''julia print_top(io, mime, A) '''

But read the DimensionalData.jl show.jl code for details.

source

# DimensionalData.show_after — Function.

show_after(io::IO, mime, A::AbstractDimArray)
show_after(io::IO, mime, A::AbstractDimStack)

Interface methods for adding addional show text for AbstractDimArray/AbstractDimStack subtypes.

Always include kw to avoid future breaking changes

Additional keywords may be added at any time.

blockwidth is passed in context

blockwidth = get(io, :blockwidth, 10000)

Note - a anssi box is left unclosed. This method needs to close it, or add more. blockwidth is the maximum length of the inner text.

Most likely you always want to at least close the show blocks with:

'''julia print_block_close(io, blockwidth) '''

But read the DimensionalData.jl show.jl code for details.

source

# DimensionalData.refdims_title — Function.

refdims_title(A::AbstractDimArray)
refdims_title(refdims::Tuple)
refdims_title(refdim::Dimension)

Generate a title string based on reference dimension values.

source