splitfile
Python file-like object that facilitates reading and writing of binary data split across multiple volumes (files) of a specified size. Supports random access IO.
Can be used with modules such as tarfile, zipfile, lzma, etc. enabling read/write of split archives.
Note: Version 2.0 removes previously integrated encryption and compression options. The breadth of choice across both suggests they are better handled as a separate layer, leaving splitfile to simply manage the final IO across volumes.
Examples
import splitfile
with splitfile.open('data.bin', 'wb') as f:
f.write(b'Hello, World!')
import splitfile
import tarfile
with splitfile.open('data.bin', 'wb', volume_size=1000000) as f:
with tarfile.open(mode='w', fileobj=f) as t:
for file in files:
t.add(file)
Result:
data.bin
data.bin.2
data.bin.3
...
Documentation
splitfile.open(filename, mode, volume_size=0, append_to_partial=True)
Returns a SplitFile object.
- filename contains a valid file path. Volumes add a suffix (.2, .3, etc.).
- Supported mode values are
wb, wb+, ab, ab+, rb, rb+
. - volume_size specifies the max size of a volume in bytes.
- append_to_partial set to True appends data written to the end of a previously created file by adding to the last volume if its size is less than volume_size. If False, a new volume is always created for writing beyond the end of an existing file.
Dependencies
None
Installation
pip install splitfile