ModelParameters

ModelParameters

ModelParameters simplifies the process of writing and using complex, high performance models, decoupling technical decisions about model structure and composition from usability concerns.

It provides linear indexing of parameters, a Tables.jl interface, and controllable Interact.jl Interfaces (via InteractModels.jl) – for any object, of any complexity. Parameters of immutable objects can be updated from a vector, tuple or table using a single command, rebuilding the object with the new values.

Use case

ModelParameters.jl is designed to help writing physics/environmental/ecological models with heterogeneous structure and multiple formulation options.

Once these models grow beyond a certain complexity it becomes preferable to organise them in modular way, and to reuse components in variants in other models. This pattern is seen in climate models and land models related to CLIMA project, and in ecological modelling tools like DynamicGrids.jl and GrowthMaps.jl that this package was built for.

Models may be structured as a composed nested hierarchy of structs, Tuple chains of objects, NameTuples, or some combination of the above. For performance, or running on GPUs, immutability is often necessary.

The problem comes when trying to use these models in Optim.jl, or run sensitivity analysis on them with DiffEqSensitivity.jl, or pass priors to a Bayesian modelling package. These packages often need parameter values, bounds and priors as Vectors. They may also need to update the model with new parameters as required. Writing out these conversions for every model combination is error prone and inefficient - especially with nested immutable models, that need to be rebuilt to change the parameters.

ModelParameters.jl can convert any arbitrarily complex model built with structs, Tuples and NamedTuples into vectors of values, bounds, priors, and anything else you need to attach, and easily reconstruct the whole model when they are updated. This is facilitated by wrapping your parameters, wherever they are in the model, in a Param:

using ModelParameters

Base.@kwdef struct Submodel1{A,B}
    α::A = Param(0.8, bounds=(0.2, 0.9))
    β::B = Param(0.5, bounds=(0.7, 0.4))
end

Base.@kwdef struct Submodel2{Γ}
    γ::Γ = Param(1e-3, bounds=(1e-4, 1e-2))
end

Base.@kwdef struct SubModel3{Λ,X}
    λ::Λ = Param(0.8, bounds=(0.2, 0.9))
    x::X = Submodel2()
end

julia> model = Model((Submodel1(), SubModel3()))
Model with parent object of type: 

Tuple{Submodel1{Param{Float64,NamedTuple{(:val, :bounds),Tuple{Float64,Tuple{Float64,Float64}}}},Param{Float64,NamedTuple{(:val, :bounds),Tuple{Float64,Tuple{Float64,Float64}}}}},SubModel3{Param{Float64,NamedTuple{(:val, :bounds),Tuple{Float64,Tuple{Float64,Float64}}}},Submodel2{Param{Float64,NamedTuple{(:val, :bounds)
,Tuple{Float64,Tuple{Float64,Float64}}}}}}}

And parameters:
┌───────────┬───────┬───────┬────────────────┐
│ component │ field │   val │         bounds │
├───────────┼───────┼───────┼────────────────┤
│ Submodel1 │     α │   0.8 │     (0.2, 0.9) │
│ Submodel1 │     β │   0.5 │     (0.7, 0.4) │
│ SubModel3 │     λ │   0.8 │     (0.2, 0.9) │
│ Submodel2 │     γ │ 0.001 │ (0.0001, 0.01) │
└───────────┴───────┴───────┴────────────────┘

julia> model[:val]
(0.8, 0.5, 0.8, 0.001)

To get the model values as a vector for Optim.jl, simply:

collect(model[:val])

What are Params?

Param is a wrapper for your parameter value and any metadata you need to track about it. Param has flexible fields, but expects to always have a :val field – which is the default if you don't used a keyword argument:

par = Param(99.0)
@assert par.val == 99.0

Internally Param uses a NamedTuple to be flexible for scripting. You can just add any fields you need. When parameters are built into a Model, they are standardised so that they all have the same fields, filling the gaps with nothing.

There are a few other "privileged" fields that have specific behaviour, if you use them. A units field will be combined other fields using withunits, and this is done by default for val when you run stripparams on the models - if there is actually a units field. The InteractModel in the sub-package InteractModels.jl may also combine range or bounds fields with units and use them to construct sliders.

Param is also a Number, and should work as-is in a lot of models for convenience. But it can easily be stripped from objects using stripparams.

What is a Model?

A model is another wrapper type, this time for a whole model - whatever it may be. Its a mutable and untyped containers for you typed, immutable models, so they can be updated in a user interface or by using setproperties!. Letting you keep a handle to the updated version. Model gives you a Tables.jl interface, provides a table of parameters in the REPL, and give you some powerful tools for making changes to your model.

There is a more limited StaticModel variant where you need maximum performance and don't need a handle to the model object.

An InteractModel from the InteractModels.jl subpackage is identical to Model, with the addition of an Interact.jl interface. It accepts a function that generates anything that can go into a web page (like a plot) in response to model parameter changes you make with the generated sliders.

Setting model values

Setting new values

You can also add new columns to all model parameters directly from the model:

model[:bounds] = ((1.0, 4.0), (0.0, 1.0), (0.0, 0.1), (0.0, 100.0))

Swapping number types

ModelParameters makes it very easy to make modifications to your model parameters. To update all model values to be Float32, you can simply do:

model[:val] = map(Float32, model[:val])

Tables.jl interface

You can also save and import your model parameters to/from CSV or any other kind of Table or DataFrame using the Tables.jl interface:

update!(model, table)

Live Interact.jl models

InteractModels.jl is a subpackage of ModelParameters.jl, but needs to be installed separately. This avoids loading the heavy web-stack dependencies of Interact.jl when you don't need them.

Using InteractModels, any model can have an Interact.jl web interface defined for it automatically, by providing a function that plots or displays your model in some way that can show in a web page. The interface, slider controllers and model updates are all taken care of.

Potential Problems

If you define structs with type parameters that are not connected to fields, ModelParameters.jl will not be able to reconstruct them with new Param values, or use stripparams to remove the Param wrappers.

Defining ConstructionBase.constructorof from ConstructionBase.jl is the solution to this, and will also mean your objects can be used with other packages for immutable manipulation like Flatten.jl, Setfield.jl, Accessors.jl and BangBang.jl.

ConstructionBaseExtras.jl also exists to add support to common packages, such as StaticArrays.jl arrays. Import it if you need StaticArrays support, or open an issue to add support to additional packages.

Note: Breaking change in 0.4.0 With the introduction of weak extensions in Julia 1.9, ConstructionBase.jl and ConstructionBaseExtras.jl should not be loaded at the same time (see this issue). ModelParameters.jl has dropped the direct dependency on ConstructionBase.jl in version 0.4.0. Users that employ Julia versions <1.9 are advised to load ConstructionBaseExtras.jl themselves if StaticArrays.jl support is needed.

Abstract supertype for model wrappers like Model, useful if you need to extend the behaviour of this package.

Accessing AbstactModel parameters

Fields can be accessed with getindex:

model = Model(obj)
@assert model[:val] isa Tuple
@assert model[:val] == model[:val]
@assert model[:units] == model[:units]

To get a combined Tuple of val and units, use withunits.

The type name of the parent model component, and the field name are also available:

model[:component]
model[:fieldname]

Getting a Vector of parameter values

Base methods collect, vec, and Array return a vector of the result of model[:val]. To get a vector of other parameter fields, simply collect the tuple:

boundsvec = collect(model[:bounds])

Tables.jl interface

All AbstractModels define the Tables.jl interface. This means their paremeters and parameter metadata can be converted to a DataFrame or CSV very easily:

df = DataFrame(model)

Tables.rows will also return all Params as a Vector of NamedTuple.

To update a model with params from a table, use update! or update:

update!(model, table)

AbstractModel Interface: Defining your own model wrappers

It may be simplest to use ModelParameters.jl on a wrapper type you also use for other things. This is what DynamicGrids.jl does with Ruleset. It's straightforward to extend the interface, nearly everything is taken care of by inheriting from AbstractModel. But in some circumstances you will need to define additional methods.

AbstractModel uses Base.parent to return the parent model object. Either use a field :parent on your <: AbstractModel type, or add a method to Base.parent.

With a custom parent field you will also need to define a method for setparent! and setparent that sets the correct field.

An AbstractModel with complicated type parameters may require a method of ConstructionBase.constructorof.

To add custom show methods but still print the parameter table, you can use:

printparams(io::IO, model)

That should be all you need to do.

Abstract supertype for parameters. Theses are wrappers for model parameter values and metadata that are returned from params, and used in getfield/setfield/getpropery/setproperty methods and to generate the Tables.jl interface. They are stripped from the model with stripparams.

An AllParams must define a Base.parent method that returns a NamedTuple, and a constructor that accepts a NamedTuple. It must have a val property, and should use checkhasval in its constructor.

MakieModel(f, model)

An AbstractModel that generates its own Makie.jl interface. Each model Param has a slider generated for it to update the model. Function f is passed a Makie.GridLayout to plot into and an Observables.Observable that holds the model object stripped of Params, with the values of the sliders in the interface. After any slider updates the Observable is updated.

Function f only runs once, on initialisation. In it Makie.lift should be used on the model to create an Observable which can be plotted into one or many Makie.Axis created in the GridLayout.

Arguments

• f: a function that take a GridLayout and model object wrapped as an Observable as arguments.
• model: any object with Params objects in some fields.

Keyword Arguments

• title: "" set a window title, if you need it.
• slider_kw: An optional NamedTuple of keywords to pass to all sliders.
• ncolumns: Group sliders in n columns, 1 by default.
• figure: An optional Makie Figure.
• layout: An optional Makie GridLayout that gets passed to f

Param fields

Param objects in the model can include keywords:

• label: field to use instead of field names
• range: an AbsractRange of slider positions
• bounds: an NTuple{2} for min and max of slider ranges, if range is not available.

Withouth range or bounds the range will be guessed from val.

Example

This is a simple example where the model is a NamedTuple that changes the color patterns of

using ModelParameters, GLMakie, CSV

# Define some parameters
model = (; 
    noise=Param(0.5, bounds=(0.0, 1.0), label="Noise"),
    color=Param(0.5, bounds=(0.0, 1.0), label="Color"),
)

# Define a function that generates a random array from our model `m`
randarray(m) = max.(min.((rand(10, 10) .- 0.5) .* m.noise .+ m.color, 1.0), 0.0)

# Make an interactive model 
mm = MakieModel(model; ncolumns=2) do layout, model_obs
    # `model_obs` is our model with `Params` stripped and wrapped as an `Observable`.
    # We can `lift` it to run `f` and update a new 
    x = lift(randarray, model_obs)
    # Define an axis to plot into
    ax = Axis(layout[1, 1])
    # And plot a heatmap of the output of `f`
    heatmap!(ax, x; colorrange=(0, 1))
end

# We can save the parameters set in the interface
# to anything Tables.jl compatible, like csv
CSV.write("modelparams.csv", mm)

Model(x)

A wrapper type for any model containing Param parameters - essentially marking that a custom struct or Tuple holds Param fields.

This allows you to index into the model as if it is a linear list of parameters, or named columns of values and paramiter metadata. You can treat it as an iterable, or use the Tables.jl interface to save or update the model to/from csv, a DataFrame or any source that implements the Tables.jl interface.

Param(p::NamedTuple)
Param(; kw...)
Param(val)

A wrapper type that lets you extract model parameters and metadata about the model like bounding val, units priors, or anything else you want to attach.

The first argument is assigned to the val field, and if only keyword arguments are used, val, must be one of them. val is used as the number val if the model us run without stripping out the Param fields. stripparams also takes only the :val field.

RealParam(p::NamedTuple)
RealParam(; kw...)
RealParam(val)

A wrapper type that lets you extract Real typed model parameters and metadata about the model like bounding val, units priors, or anything else you want to attach.

The first argument is assigned to the val field, and if only keyword arguments are used, val, must be one of them. val is used as the number val if the model us run without stripping out the Param fields. stripparams also takes only the :val field.

StaticModel(x)

Like Model but immutable. This means it can't be used as a handle to add columns to your model or update it in a user interface.

component(::Type{T}) where T

Generates the identifier stored in the :component field of an AbstractModel. The default implementation simply uses T.name.wrapper which is the UnionAll type corresponding to the unparameterized type name of T.

groupparams(m::AbstractModel, cols::Symbol...)

Groups parameters in m hierarchically according to cols. A Symbol constructor must be defined for the value type of each parameter field (e.g. String, Symbol, and Int would all be valid by default). The returned value is a nested named tuple where the hierachical order follows the order of cols.

For example, we could group parameters first by component name, then by field name:

Examples

julia> groupparams(Model((a=Param(1.0), b=Param(2.0))), :component, :fieldname)
(NamedTuple = (a = ..., b = ...),)

mapflat(f, collection; maptype::Type=Union{NamedTuple,Tuple,AbstractArray})

"Flattened" version of map where f is applied to all nested non-collection elements of x. The transformed result is returned with the nested structure of the input x unchanged. Note that this differs from flatmap in functional settings, which is typically just map followed by flatten.

Examples

julia> mapflat(x -> 2*x, (a = (b = (1,)), c = (d = (2,))))
(a = (b = (2,)), c = (d = (4,)))

params(object)
params(model::AbstractModel)

Returns a tuple of all Params in the model or arbitrary object.

printparams(object)
printparams(io::IO, object)

Prints a table of all Params in the object, similar to what is printed in the repl for AbstractModel.

setparent(model::AbstractModel, x)

Internal interface method to define for custom AbstractModel with a different field for parent.

Set the parent object and return the rebuilt model. Must be defined if the parent field of an AbstractModel is not :parent.

setparent!(model::MutableModel, x)

Internal interface method to define for custom AbstractModel with a different field for parent.

Set the parent object. Must be defined if the parent field of an AbstractModel is not :parent.

stripparams(object)

Strips all AbstractParam from an object, replacing them with the val field, or a combination of val and units if a units field exists.

stripunits(model::AbstractModel, xs)
stripunits(param::AbstractParam, x)

Returns the x or xs divided by their corresponding units field, if it exists.

It there is no units field, and x has units, it will be returned with units! It you want to simply remove all units, using Unitful.ustrip.

update(m::AbstractModel, table)

Update the model from an object that implements the Tables.jl interface, returning a new, updated object.

update!(m::MutableModel, table)

Update the model in-place from an object that implements the Tables.jl interface.

Note: the parent object can be immutable, it will be completely rebuilt. But the wrapper AbstractModel is mutable, such as Model or InteractModel.

withunits(object, [fieldname])
withunits(model::AbstractModel, [fieldname])
withunits(param::AbstractParam, [fieldname])

Returns the field specifed by fieldname (by default :val) for a single Param, or a tuple of the Params in a Model or arbitrary object.

If there is a units field the returned value will be a combination of the specied field and the units fields.

If there is no units field or a specific Params units fields contains nothing, the field value is returned unchanged.