Metadata

Metadata.jl

Introduction

The term "metadata" is widely used across very different applications. Therefore, "metadata" may translate to very different structures and implementations in code. The Metadata package attempts to provide a generic interface for interacting with metadata in Julia that is agnostic to the exact type of metadata present. This package typically assumes metadata to be a collection of values paired to Symbol keys (e.g., AbstractDict{Symbol,Any}, NamedTuple), but metadata that doesn't perfectly fit this criteria should still work with most methods if adhering to the basic interface.

Attaching Metadata

The most important method to know is attach_metadata. It's intended to give users a generic way of attaching metadata to any given type without worrying about the particulars what type is appropriate for binding metadata to a particular. For example, attaching metadata to an array should produce something that can act like an array still. Instead of requiring users to know what type is used internally (Metadata.MetaArray), an appropriate type is chosen by default and the method of accessing metadata is the same.

julia> using Metadata

julia> x = ones(2, 2);

julia> meta = (x = 1, y = 2);

julia> mx = attach_metadata(x, meta)
2×2 attach_metadata(::Array{Float64,2}, ::NamedTuple{(:x, :y),Tuple{Int64,Int64}}
  • metadata:
     x = 1
     y = 2
)
 1.0  1.0
 1.0  1.0

julia> mx.x
1

julia> mx.y
2

julia> attach_metadata(x, (x = 1, y = 2, suppress= [:x]))
2×2 attach_metadata(::Array{Float64,2}, ::NamedTuple{(:x, :y, :suppress),Tuple{Int64,Int64,Array{Symbol,1}}}
  • metadata:
     x = <suppressed>
     y = 2
)
 1.0  1.0
 1.0  1.0

There are three things you should notice from the previous example:

1. The display is nearly identical to how the parent x would be printed. The only addition is a list of the metadata and the argument used to bind the x and meta.
2. We can access the metadata as if they were properties.
3. We can suppress the printing of any value if metadata(x, :suppress) returns a collection of symbols containing that value.

There are a limited number of interfaces that require special types for binding metadata. The rest are bound to Metadata.MetaStruct.

julia> mr = attach_metadata(3//5, meta)
attach_metadata(3//5, ::NamedTuple{(:x, :y),Tuple{Int64,Int64}})
  • metadata:
     x = 1
     y = 2

julia> propertynames(mr)
(:x, :y)

julia> mr.num
3

julia> mr.den
5

Here we attached the same metadata to a rational number. Again, our metadata is now considered the properties of mr, but we can still access the parent's properties.

If the type you want to attach metadata to is mutable then each instance has a unique global identifier and you may attach metadata to a global dictionary.

julia> x = ones(2, 2);

julia> @attach_metadata(x, meta);

julia> @metadata!(x, :z, 3);

julia> @metadata(x, :z)
3

julia> Pair(:x, 1) in @metadata(x)
true

If users want to access all of the metadata from one structure and attach it to another they should instead use share_metadata(src, dst) or copy_metadata(src, dst).

julia> mx = attach_metadata(ones(2, 2), @metadata(x));

julia> mx2 = share_metadata(mx, ones(2, 2));

julia> metadata(mx2) === metadata(mx)
true

julia> mx3 = copy_metadata(mx2, ones(2, 2));

julia> metadata(mx3) === metadata(mx2)
false

julia> metadata(mx3) == metadata(mx2)
true

Interface For Specific Metadata Glue Types

A new structure of type T and instance g that glues some data x to metadata m require the following methods:

attach_metadata(x, metadata)

Generic method for attaching metadata to x.

@attach_metadata(x, meta)

Attach metadata meta to the object id of x (objectid(x)) in the current module's global metadata.

See also: GlobalMetadata

has_metadata(x[, k; dim]) -> Bool

Returns true if x has metadata. If k is specified then checks for the existence of a metadata paired to k. If dim is specified then this checks the metadata at the corresponding dimension.

@has_metadata(x) -> Bool
@has_metadata(x, k) -> Bool

Does x have metadata stored in the curren modules' global metadata? Checks for the presenece of the key k if specified.

metadata(x[, k; dim])

Returns metadata from x. If k is specified then the metadata value paired to k is returned. If dim is specified then the operation is performed for metadata specific to dimension dim.

@metadata(x[, k])

Retreive metadata associated with the object id of x (objectid(x)) in the current module's global metadata. If the key k is specified only the value associated with that key is returned.

metadata!(x, k, val[; dim])

Set x's metadata paired to k to val. If dim is specified then the metadata corresponding to that dimension is mutated.

@metadata!(x, k, val)

Set the value of x's global metadata associated with the key k to val.

copy_metadata(src, dst) -> attach_metadata(dst, copy(metadata(src)))

Copies the the metadata from src and attaches it to dst. Note that this method specifically calls deepcopy on the metadata of src to ensure that changing the metadata of dst does not affect the metadata of src.

See also: share_metadata.

@copy_metadata(src, dst) -> attach_metadata(dst, copy(metadata(src)))

Copies the metadata from src by attaching it to dst. This assumes that metadata for src is stored in a global dictionary (i.e. not part of src's structure) and attaches a new copy to dst through a global reference within the module.

See also: @share_metadata, copy_metadata

share_metadata(src, dst) -> attach_metadata(dst, metadata(src))

Shares the metadata from src by attaching it to dst. The returned instance will have properties that are synchronized with src (i.e. modifying one's metadata will effect the other's metadata).

See also: copy_metadata.

@share_metadata(src, dst) -> @attach_metadata(@metadata(src), dst)

Shares the metadata from src by attaching it to dst. This assumes that metadata for src is stored in a global dictionary (i.e. not part of src's structure) and attaches it to dst through a global reference within the module.

See also: @copy_metadata, share_metadata

drop_metadata(x)

Returns x without metadata attached.

test_wrapper(::Type{WrapperType}, x::X)

Tests the metadata interface for a metadata wrapper (WrapperType) for binding instances of type X. It returns the results of attach_metadata(x, Dict{Symbol,Any}()) for further testing.

NoMetadata

Internal type for the Metadata package that indicates the absence of any metadata. DO NOT store metadata with the value NoMetadata().

metadata_summary([io], x)

Creates summary readout of metadata for x.

MetaArray(parent::AbstractArray, metadata)

Custom AbstractArray object to store an AbstractArray parent as well as some metadata.

Examples

julia> using Metadata

julia> Metadata.MetaArray(ones(2,2), metadata=(m1 =1, m2=[1, 2]))
2×2 attach_metadata(::Array{Float64,2}, ::NamedTuple{(:m1, :m2),Tuple{Int64,Array{Int64,1}}}
  • metadata:
     m1 = 1
     m2 = [1, 2]
)
 1.0  1.0
 1.0  1.0

MetaRange(x::AbstractRange, meta)

Type for storing metadata alongside a range.

Examples

julia> using Metadata

julia> Metadata.MetaRange(1:1:2, (m1 =1, m2=[1, 2]))
attach_metadata(1:1:2, ::NamedTuple{(:m1, :m2),Tuple{Int64,Array{Int64,1}}})
  • metadata:
     m1 = 1
     m2 = [1, 2]

MetaUnitRange(x::AbstractUnitRange, meta)

Type for storing metadata alongside a anything that is subtype of AbstractUnitRange.

Examples

julia> using Metadata

julia> Metadata.MetaUnitRange(1:2, (m1 =1, m2=[1, 2]))
attach_metadata(1:2, ::NamedTuple{(:m1, :m2),Tuple{Int64,Array{Int64,1}}})
  • metadata:
     m1 = 1
     m2 = [1, 2]

MetadataPropagation(::Type{T})

Returns type informing how to propagate metadata of type T. See DropMetadata, CopyMetadata, ShareMetadata.

CopyMetadata

Informs operations that may propagate metadata to attach a copy to any new instance created.

DropMetadata

Informs operations that may propagate metadata to insead drop it.

ShareMetadata

Informs operations that may propagate metadata to attach a the same metadata to any new instance created.