Combining data and metadata: hybrid tabular file formats

PO
Not scheduled
15m
Wichernhaus

Wichernhaus

Board: L196
poster presentation Lessons learned Poster

Speaker

Mark Taylor (University of Bristol)

Description

When working with astronomical data, metadata is also important. A general-purpose file format for transmission, processing and archiving large datasets should facilitate, among other things, both efficient processing of bulk data and encoding of rich semantic metadata. When choosing a format for a particular purpose sometimes no existing format satisfies both these requirements adequately, but combining one data-efficient and one metadata-rich format can be made to do so.

This poster discusses considerations for designing such hybrid data/metadata formats, and lists some examples. We focus on tabular data, but some of the considerations may apply to other datatypes such as arrays as well.

Affiliation of the submitter University of Bristol
Attendance in-person

Primary author

Mark Taylor (University of Bristol)

Presentation materials