PyRabin
A python module for generating rabin fingerprints.
The C extension implementation is based on rabin-fingerprint-c with some bugs fixed.
Features
- Fast fingerprint generation.
- Split file into blocks according to the rabin fingerprints.
- Window/Chunk size is configurable.
API
Module methods
-
set_prime(integer)
- Set Rabin polynomial prime, default is 3.
-
set_window_size(integer)
- Set Rabin polynomial sliding window size, default is 48 bytes.
-
set_max_block_size(integer)
- Set Rabin polynomial max block size, default is 64K bytes.
-
set_min_block_size(integer)
- Set Rabin polynomial min block size, default is 2K bytes
-
set_average_block_size(integer)
- Set Rabin polynomial average block size, average is 8K bytes.
-
get_file_fingerprints(filename)
- Returns a list of tupple, each tuple represents (offset, length, fingerprint)
-
split_file_by_fingerprints(filename)
- Returns a list of tupple, each tuple represents (offset, length, fingerprint, filename).
- The default filename is sha1sum of the chunk with '.blk' suffix.
class Rabin
-
register(callback)
- Register a callback for calling when a block is reached
- The callback is called with three parameters: (offset, length, fingerprint)
-
update(data)
- Update Rabin fingerprint list by adding a block of data
-
clear()
- Clear fingerprints and reset status
-
fingerprints()
- Return fingerprints of all data passed by the update method
Examples
- See test/test.sh for example
Bugs
- Please report bugs / mis-formatted pages to the github issure tracker.
Contributing
- Fork it
- Create your feature branch (git checkout -b my-new-feature)
- Commit your changes (git commit -am 'Add some feature')
- Push to the branch (git push origin my-new-feature)
- Create new Pull Request