pykleio

Versioned Storage Python SDK - enable version management for nd data


License
MIT
Install
pip install pykleio==0.5.1

Documentation

Versioned Storage Python SDK

| JAVA SDK IS HERE


Proposed solution:

To enable block-based data versioning for nd data, a mix is created of:

  • Version block index using Zarr + Git
  • A key value store: using N5 for now

How to:

index_store = ZarrIndexStore(INDEX_PATH)
store = VersionedFSStore(index_store, RAW_PATH)

z = zarr.open(store, mode="a")

# Commit
store.vc.commit()

# Push
store.vc.push()

# Create new branch
store.vc.create_new_branch(BRANCH_NAME)

# Checkout new branch
store.vc.create_new_branch(BRANCH_NAME)

Features:

  • Multiple branches
  • Multiple collaborators
  • Can jump anytime to any historical point
  • Data is not replicated and no extra reading writing cost

Every change is a now commit:

solution

Commits