epHDF – a proposed standard for storing electrophysiology data in HDF5
-
1
University of California at Berkeley, United States
The HDF5 file format is becoming increasingly popular for storing scientific data including electrophysiology data. The efficient sharing of electrophysiology data using HDF5 will require conventions for how the data are organized within HDF5 files. The determination of such conventions is difficult for at least two reasons. First, it is virtually impossible to anticipate all of the types of data and metadata that will need to be stored. Second, no standard scheme exists for specifying how data in HDF5 files should be organized.
To address both of these difficulties, we propose a layered approach. The first layer, which we call “HDFds” (for “Hierarchical Data Format – data sharing”), provides domain-independent conventions for specifying how the data in HDF5 files are organized. Main features of HDFds: a) Enables associating external schemata to components of an HDF5 file in a manner similar to how name spaces in an XML file identify elements. b) Specifies locations and a format for storing arbitrary metadata in a HDF5 file. c) Allows linking metadata to particular data parts within a file and to external files.
The second layer builds on the conventions in HDFds to specify schemata for storing basic electrophysiology data types. We call this second layer “epHDF”, for “electrophysiology HDF”. The data types defined in epHDF (time series, time series segment, neural event and experimental event) are based on the entities defined in Neuroshare for covering the most commonly used data types in electrophysiology. For each type, the data can be stored in whatever HDF5 numeric format is most efficient (for example 16 bit integer). For all of the data types, a metadata schema is specified that includes the fields needed to make a plot of the data with correct units.
epHDF does not constrain the location of entities within the HDF5 file and allows defining and reference additional schemata to add new capabilities. This flexibility enables constructing new conventions as needed while still maintaining the capability to interpret the basic electrophysiology data types required for data sharing.
A test was done by converting a sample file, provided by a recording equipment manufacture, from a custom HDF5 format to epHDF. Only minor changes were required to do the conversion and the file size was reduced. The use cases presented in the poster suggest that the epHDF format is simple and also efficient.
Acknowledgements
This work was conducted within the Electrophysiology Task Force of the INCF Program on Standards for Data Sharing. Funding for J.L. Teeters and F.T. Sommer provided through NSF grant 0855272.
References
Keywords:
HDF5,
data sharing,
Electrophysiology,
Standards,
File format
Conference:
Neuroinformatics 2013, Stockholm, Sweden, 27 Aug - 29 Aug, 2013.
Presentation Type:
Poster
Topic:
Electrophysiology
Citation:
Teeters
JL and
Sommer
FT
(2013). epHDF – a proposed standard for storing electrophysiology data in HDF5.
Front. Neuroinform.
Conference Abstract:
Neuroinformatics 2013.
doi: 10.3389/conf.fninf.2013.09.00068
Copyright:
The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers.
They are made available through the Frontiers publishing platform as a service to conference organizers and presenters.
The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated.
Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed.
For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions.
Received:
29 Apr 2013;
Published Online:
11 Jul 2013.
*
Correspondence:
Dr. Jeffrey L Teeters, University of California at Berkeley, Berkeley, United States, teeters@berkeley.edu