nimhdf5

Bindings for the HDF5 data format C library


Keywords
library, wrapper, binding, libhdf5, hdf5, ndarray, storage
License
MIT
Install
nimble install nimhdf5

Documentation

nimhdf5

This repository contains a wrapper for the HDF5 data format for the Nim programming language as well as some very work in progress high-level bindings.

The raw wrapper dynamically links the libhdf5.so (the main library) and libhdf5_hl.so (a library containing high-level convenience functions) libraries at runtime. All public functions of the two libraries are callable by their corresponding C names and C arguments. That means the Nim datatypes need to be manually cast to the corresponding compatible types, if only the wrapper is used.

This is currently in a pre-alpha state, with no guarantees that it will work for you. The wrapper was built making heavy use of c2nim and the main goal for now was to get the code to compile and be able to call basic HDF5 functions like opening and closing HDF5 files, creating a simple dataset and reading it. More advanced HDF5 features have not been tested. The functions are accessible, but may potentially be completely broken.

The high-level bindings are now at a point where most basic operations sort of work. See examples/h5_create_dataset_hl.nim as an example of writing arbitrary groups and datasets and reading them back.

Compatibility

The wrapper is currently tested using Nim version 0.17.2 and the current devel branch (0.17.3). The wrapper is built from HDF5 version 1.10.1.

Linking against the HDF5 1.8 library is reasonably supported as well, but requires to use an additional compiler flag for now:

-d:H5_LEGACY

Currently no checks are done, which compare the library this wrapper is built upon with the library linking against using some of the provided HDF5 macros (e.g. H5check, H5get_libversion etc.). The main reason is explained below. However, as far as the high-level functionality is concerned at the moment, the only differences arise in a few constant definitions, whose names slightly changed from 1.8 to 1.10. This is what the compiler flags sets accordingly.

The HDF5 headers contain macros for many variables, such as

#define H5F_ACC_RDONLY	(H5CHECK H5OPEN 0x0000u)

where

#define H5CHECK          H5check(),

and

#define H5OPEN        H5open(),

i.e. it makes use of C’s comma operator. However, c2nim currently has no support for it. Instead of porting them in some reasonable way, these macros were converted to simple replacements with the values, dropping the calls to H5check() and H5open().

The call to H5check() is currently not used at all. Compiling a Nim program with this wrapper (based on version 1.10.1) would normally fail to check against the linked library, if that version is different.

As H5open() is important, the calls are replaced by a single call to initialize the library at the beginning upon the first call of the library via src/nimhdf5/wrapper/H5niminitialize.nim.

As HDF5 is a very macro heavy library, other important macros may not have been correctly wrapped to Nim, e.g. determination of correct sizes of data types. This may cause some weird side-effects.

Additionally, Windows is currently not supported. In principle, adding support might be as easy as changing the name of libhdf5.so to libhdf5.dll (?) as well as wrapping H5FDwindows.h.

Installation

As the state of the library is still in such an early stage, it’s not yet added to the nimble repository. You can clone this repository manually

git clone https://github.com/vindaar/nimhdf5

in a folder of your choice and call nimble install afterwards:

cd nimhdf5
nimble install

Dependencies

Since I personally need to write arraymancer tensors into HDF5 files, the requirement of arraymancer is currently hardcoded, despite not being needed for the library.

Files

The folder c_headers contains the modified HDF5 headers in the state they were in for a successful c2nim conversion. In some cases the C header file had to be modified, in others modification to the resulting .nim file was still necessary.

The folder examples contains the basic HDF5 C examples (see here: https://support.hdfgroup.org/HDF5/examples/intro.html#c) converted to Nim utilizing the wrapper. h5_create_dataset_hl.nim is an example, which uses the currently available high-level bindings.

To be done

  • build unit tests
  • add reading of hyperslabs
  • do more clean up of code
  • rewrite parts making better use of available H5 functions
  • … lots and lots more