npy-cpp-patches
Read N-Dimensional patches from .npy files. This module is built using C++ and has Python3 bindings.
Data Specifications
- Arrays must be saved in
C-contiguousformat, i.e. NOTFortran-contiguous. - First dimension is indexed using in a non-contiguous manner. For example, this can be used to extract specific channels within a natural image.
- Next dimensions are specified by a patch shape
C++vector orPythontuple. To extract patches of lower dimensionality than that of the data, set the corresponding dimensions to1.
Python Usage
Install
pip install npy-patcher
Usage
from npy_patcher import PatcherDouble, PatcherFloat, PatcherInt, PatcherLong
# In this example lets say our data has shape (10, 90, 90), and is therefore 3D.
data_fpath = '/my/numpy/file.npy'
patcher = PatcherFloat() # Use PatcherFloat for np.float32 datatype
nc_index = [0, 1, 3, 5, 7] # Non-contiguous index
patch_shape = (30, 30) # Contiguous patch shape.
patch_stride = (30, 30) # Contiguous patch stride, set a smaller value than `patch_stride` for overlapping patches.
patch_num = 2
# The patch number indexes the patches (starting from 0). So in our example the index 2
# would be equivalent to data[nc_index, 0:30, 60:90]. The variable indexes the patches
# in C-contiguous manner, i.e. the last dimension has the smallest stride.
extra_padding = (0, 0) # Optionally apply extra padding, this is applied after initial padding calculation.
patch_num_offset = (0, 0) # Optionally provide an offset to extract patches after.
patch = patcher.get_patch(
data_fpath, nc_index, patch_shape, patch_stride, patch_num, extra_padding, patch_num_offset
)
patch = patch.reshape((5, 30, 30)) # PatcherFloat returns a list, therefore we need to reshape.C++ Usage
Below is an example written in C++, equivalent to the Python usage above.
// test.cpp
#include "src/patcher.hpp"
#include <vector>
#include <string>
int main() {
std::string fpath = "data.npy";
std::vector<size_t> nc_index {0, 1, 3, 5, 7};
std::vector<size_t> patch_shape {30, 30};
std::vector<size_t> patch_stride{30, 30};
size_t patch_num = 2;
std::vector<size_t> extra_padding = {0, 0};
std::vector<size_t> patch_num_offset = {0, 0};
Patcher<float> patcher;
// Here the patch object is a contiguous 1D vector
std::vector<float> patch = patcher.get_patch(
fpath, nc_index, patch_shape, patch_stride, patch_num, extra_padding, patch_num_offset
);
return 0;
}You can then build the package, for example using g++:
$ cd npy-cpp-patches/
$ g++ -std=c++17 -I ./ -g test.cpp src/npy_header.cpp src/pyparse.cpp -o test
