Integrating data storage and annotation in the data workflow using the NIX format and libraries
-
1
Ludwig-Maximilians-Universität München, German Neuroinformatics Node, Germany
-
2
Universität Tübingen, Institut für Neurobiologie, Germany
Increasing complexity of experimental approaches in neurosciences challenges methods for managing recorded data and metadata. Storing such information consistently is an essential part of experimental research and depends crucially on available file formats. Currently existing file formats are subject to several restrictions: some formats are vendor specific or only accessible via proprietary software. Others are highly domain specific, designed with respect to efficiency for certain kinds of data and therefore not versatile enough to be used in a wide variety of use cases. Moreover, many existing formats provide only limited support for storing metadata along with the data. A common, open file format that is versatile enough to represent various kinds of data in conjunction with metadata has the potential to increase community-based tool development as well as data sharing. The emergence of initiatives like the Electrophysiology Task Force of the INCF Data Sharing Program [1] or the NWB project [2] underlines the need for such a standardized and open file format.
The NIX project [3] specifies such a format for neuroscientific data and provides libraries for accessing these files from different platforms. The NIX format is compliant with the INCF requirements for storing electrophysiology data [1]. It is based on a well defined data model [4] which can be used to represent both data and related metadata. In particular, it provides generic entities designed to store a wide variety of data types like continuous signals, spike events, image stacks, or other multi-dimensional data. Central feature is the representation of data arrays together with units and dimension descriptors, so that the stored data can be readily interpreted as recorded quantities. The data model further defines mechanisms to specify relationships between the data arrays and to describe points or regions of interest, such as areas in an image or events in a continuous signal, supporting direct access to the referenced part of the data and the linking of metadata. Based on the data model we defined a schema for HDF5 files [5] as default format. While it is possible to read and write these files using the standard HDF5 libraries, efficient use of the features of the NIX format is facilitated by specific I/O libraries provided for different languages. The C++ library [3] serves as reference implementation and basis for interfaces to other languages. Python bindings [6] provide access to the library functionality in pythonic fashion and a quick entry point to get familiar with the concepts of the format, including detailed documentation, many examples, and tutorials [7]. Matlab bindings [8] bring the benefits of the NIX features to users of this popular tool. In addition, Java bindings are under way. The NIX file format supports comprehensive annotation and efficient organization of neuroscience data, and the variety of libraries makes it easy to integrate access to data and metadata in the lab data collection and analysis workflow.
Acknowledgements
Supported by the German Federal Ministry of Education and Research (Grant 01GQ1302)
References
[1] https://incf.org/activities/our-programs/datasharing
[2] https://crcns.org/NWB
[3] https://github.com/G-Node/nix
[4] Stoewer et al (2014) doi: 10.3389/conf.fninf.2014.18.00027
[5] http://hdfgroup.org/HDF5/
[6] https://github.com/G-Node/nixpy
[7] http://g-node.github.io/nixpy
[8] https://github.com/G-Node/nix-mx
Keywords:
HDF5,
python,
C++,
File format,
MATLAB,
Windows,
Linux,
Mac OS X,
Electrophysiology
Conference:
Neuroinformatics 2015, Cairns, Australia, 20 Aug - 22 Aug, 2015.
Presentation Type:
Demo, not to be considered for oral presentation
Topic:
General neuroinformatics
Citation:
Stoewer
A,
Kellner
CJ,
Sobolev
A,
Sonntag
M,
Benda
J,
Wachtler
T and
Grewe
J
(2015). Integrating data storage and annotation in the data workflow using the NIX format and libraries.
Front. Neurosci.
Conference Abstract:
Neuroinformatics 2015.
doi: 10.3389/conf.fnins.2015.91.00050
Copyright:
The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers.
They are made available through the Frontiers publishing platform as a service to conference organizers and presenters.
The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated.
Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed.
For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions.
Received:
13 Apr 2015;
Published Online:
05 Aug 2015.
*
Correspondence:
Mr. Adrian Stoewer, Ludwig-Maximilians-Universität München, German Neuroinformatics Node, Munich, Germany, adrian@stoewer.me