Static

Metadata.jl

Introduction

The term "metadata" is widely used across very different applications. Therefore, "metadata" may translate to very different structures and implementations in code. The Metadata package attempts to provide a generic interface for interacting with metadata in Julia that is agnostic to the exact type of metadata present. This package typically assumes metadata to be a collection of values paired to Symbol keys (e.g., AbstractDict{Symbol,Any}, NamedTuple), but metadata that doesn't perfectly fit this criteria should still work with most methods if adhering to the basic interface.

Attaching Metadata

The most important method to know is attach_metadata. It's intended to give users a generic way of attaching metadata to any given type without worrying about the particulars what type is appropriate for binding metadata to a particular. For example, attaching metadata to an array should produce something that can act like an array still. Instead of requiring users to know what type is used internally (Metadata.MetaArray), an appropriate type is chosen by default and the method of accessing metadata is the same.

julia> using Metadata

julia> x = ones(2, 2);

julia> meta = (x = 1, y = 2);

julia> mx = attach_metadata(x, meta)
2×2 attach_metadata(::Matrix{Float64}, ::NamedTuple{(:x, :y), Tuple{Int64, Int64}}
  • metadata:
     x = 1
     y = 2
)
 1.0  1.0
 1.0  1.0

julia> mx.x
1

julia> mx.y
2

julia> attach_metadata(x, (x = 1, y = 2, suppress= [:x]))
2×2 attach_metadata(::Matrix{Float64}, ::NamedTuple{(:x, :y, :suppress), Tuple{Int64, Int64, Vector{Symbol}}}
  • metadata:
     x = <suppressed>
     y = 2
)
 1.0  1.0
 1.0  1.0

There are three things you should notice from the previous example:

1. The display is nearly identical to how the parent x would be printed. The only addition is a list of the metadata and the argument used to bind the x and meta.
2. We can access the metadata as if they were properties.
3. We can suppress the printing of any value if metadata(x, :suppress) returns a collection of symbols containing that value.

There are a limited number of interfaces that require special types for binding metadata. The rest are bound to Metadata.MetaStruct.

julia> mr = attach_metadata(3//5, meta)
attach_metadata(3//5, ::NamedTuple{(:x, :y), Tuple{Int64, Int64}})
  • metadata:
     x = 1
     y = 2

julia> propertynames(mr)
(:x, :y)

julia> mr.num
3

julia> mr.den
5

Here we attached the same metadata to a rational number. Again, our metadata is now considered the properties of mr, but we can still access the parent's properties.

Creating New Metadata Types

This package creates a very minimal number of dedicated structures and creating new dedicated structures that use this interface is encouraged.

abstract type AbstractNoop end

struct Noop <: AbstractNoop end

struct MetaNoop{P<:AbstractNoop,M} <: AbstractNoop
    parent::P
    metadata::M
end

Metadata.metadata(x::MetaNoop) = getfield(x, :metadata)
Metadata.unsafe_attach_metadata(x::AbstractNoop, m) = MetaNoop(x, m)
Metadata.metadata_type(::Type{MetaNoop{P,M}}) where {P,M} = M

ArrayInterface.parent_type(::Type{MetaNoop{P,M}}) where {P,M} = P
Base.parent(x::MetaNoop) = getfield(x, :parent)

It's advised that Metadata.test_wrapper(MetaNoop, Noop()) is run to ensure it works. Note that using the dot operator (.) that aliases getproperty and setproperty! is not necessary.

CopyMetadata

Informs operations that may propagate metadata to attach a copy to any new instance created.

DropMetadata

Informs operations that may propagate metadata to insead drop it.

MDList(first::MDNode, tail::Union{MDList,NoData})

Iterable list of metadata.

MDNode(key, data)

Dedicated type for associating metadata with key. This is used to selectively reach metadata using MDNodedata.getmeta(x, key, d). key must be of a singleton type.

MetaIO(io, meta)

Type for storing metadata alongside subtypes of IO.

MetaStruct(p, m)

Binds a parent instance (p) to some metadata (m). MetaStruct is the generic type constructed when attach_metadata(p, m) is called.

See also: attach_metadata, attach_eachmeta

MetaUnitRange(x::AbstractUnitRange, meta)

Type for storing metadata alongside anything that is subtype of AbstractUnitRange. It is not intended that this be constructed directly. attach_metadata(::AbstractUnitRange, meta) should be used instead.

MetadataPropagation(::Type{T})

Returns type informing how to propagate metadata of type T. See DropMetadata, CopyMetadata, ShareMetadata.

NoData

Internal type for the Metadata package that indicates the absence of any metadata. DO NOT store metadata with the value NoData().

ShareMetadata

Informs operations that may propagate metadata to attach a the same metadata to any new instance created.

attach_metadata(x, m)

Generic method for attaching metadata m to data x. This method acts as an intermediate step where compatability between x and m is checked using checkmeta. unsafe_attach_metadata is subsequently used to quickly bind the two without and further checks.

Extended help

In general, it is not advised to define new attach_metadata methods. Instead, unique types for binding x to metadata should define a new unsafe_attach_metadata method. For example, attaching metadata to AbstractArray types by defining a unique method for unsafe_attach_metadata(x::AbstractArray, m).

See also: unsafe_attach_metadata, checkmeta

checkmeta([Type{Bool}], x, m)

Checks if the metadata m is compatible with x. If Bool is not included then an error is throw on failure.

copy_metadata(src, dst) -> attach_metadata(dst, copy(metadata(src)))

Copies the the metadata from src and attaches it to dst. Note that this method specifically calls deepcopy on the metadata of src to ensure that changing the metadata of dst does not affect the metadata of src.

See also: share_metadata.

drop_metadata(x)

Returns x without metadata attached.

getmeta(x, key, default)

Return the metadata associated with key, or return default if key is not found.

getmeta(f::Function, x, key)

Return the metadata associated with key, or return f(x) if key is not found. Note that this behavior differs from Base.get(::Function, x, keys) in that getmeta passes x to f as an argument (as opposed to f()).

has_metadata(x, k)::Bool

Returns true if metadata associated with x has the key k.

has_metadata(x)::Bool

Returns true if x has metadata.

metadata(x)

Returns metadata immediately bound to x. If no metadata is bound to x then Metadata.no_data is returned.

metadata_summary([io], x)

Creates summary readout of metadata for x.

metadata_type(::Type{T})

Returns the type of the metadata associated with T.

Metadata.metakey(m)

Returns the key associated withe the metadata m. The only way to attach a key to metadata is through MDNode(key, m).

properties(x)

Returns properties associated with bound to x.

Metadata.rmkey(m::MDNode) -> parent(m)
Metadata.rmkey(m) -> m

Returns the the metadata key associated bound to m, if m is MDNode. This is only intended for internal use.

share_metadata(src, dst) -> attach_metadata(dst, metadata(src))

Shares the metadata from src by attaching it to dst. The returned instance will have properties that are synchronized with src (i.e. modifying one's metadata will effect the other's metadata).

See also: copy_metadata.

stripmeta(x) -> (data, metadata)

Returns the the data and metadata immediately bound to x.

test_wrapper(::Type{WrapperType}, x::X)

Tests the metadata interface for a metadata wrapper (WrapperType) for binding instances of type X. It returns the results of attach_metadata(x, Dict{Symbol,Any}()) for further testing.

unsafe_attach_metadata(x, m)

Attaches metadata m to x without checking for compatability. New types for wrapping binding metadata to x should usually define a unique unsafe_attach_metadata method.

See also attach_metadata