fast base16384 encode and decode


Keywords
encode, decode, base16384, cffi, cython, pypy, pypy3, python, python-3, utf16be
License
GPL-3.0
Install
pip install pybase16384==0.3.7

Documentation

✨ pybase16384 ✨

The python binding for base16384

一种神奇的编码

pypi python implementation wheel license action

使用

  • 编码/解码文本
>>> import pybase16384 as pybs
>>> pybs.encode_string('hello!!')
'栙擆羼漡'
>>> pybs.decode_string('栙擆羼漡')
'hello!!'
  • 编码文件
from io import BytesIO

import pybase16384 as pybs

with open("input.pcm", "rb") as f:
    data = f.read()
for i in range(1):
    pybs.encode_file(BytesIO(data), open("output2.pcm", 'wb'), True)
  • 解码文件
from io import BytesIO

import pybase16384 as pybs

with open("output2.pcm", "rb") as f:
    data = f.read()
for i in range(1):
    pybs.decode_file(BytesIO(data), open("input2.pcm", 'wb'))

公开函数

from typing import IO

def encode_len(dlen: int) -> int: ...

def decode_len(dlen: int, offset: int) -> int: ...

ENCBUFSZ: int
DECBUFSZ: int
FLAG_NOHEADER: int
FLAG_SUM_CHECK_ON_REMAIN: int
FLAG_DO_SUM_CHECK_FORCELY: int

def is_64bits() -> bool: ...

def encode_file(input: IO, output: IO, write_head: bool = ..., buf_rate: int = ...): ...

def encode_file_safe(input: IO, output: IO, write_head: bool = ..., buf_rate: int = ...): ...

def decode_file(input: IO, output: IO, buf_rate: int = ...): ...

def decode_file_safe(input: IO, output: IO, buf_rate: int = ...): ...

def ensure_bytes(inp) -> bytes: ...

def encode_local_file(inp, out) -> None: ...

def decode_local_file(inp, out) -> None: ...

def encode_fd(inp: int, out: int) -> None: ...

def decode_fd(inp: int, out: int) -> None: ...

def encode_local_file_detailed(inp, out, flag: int) -> None: ...

def decode_local_file_detailed(inp, out, flag: int) -> None: ...

def encode_fd_detailed(inp: int, out: int, flag: int) -> None: ...

def decode_fd_detailed(inp: int, out: int, flag: int) -> None: ...

def encode(data: bytes) -> bytes: ...

def encode_safe(data: bytes) -> bytes: ...

def decode(data: bytes) -> bytes: ...

def decode_safe(data: bytes) -> bytes: ...

def encode_from_string(data: str, write_head: bool = ...) -> bytes: ...

def encode_from_string_safe(data: str, write_head: bool = ...) -> bytes: ...

def encode_to_string(data: bytes) -> str: ...

def encode_to_string_safe(data: bytes) -> str: ...

def encode_string(data: str) -> str: ...

def encode_string_safe(data: str) -> str: ...

def decode_from_bytes(data: bytes) -> str: ...

def decode_from_bytes_safe(data: bytes) -> str: ...

def decode_from_string(data: str) -> bytes: ...

def decode_from_string_safe(data: str) -> bytes: ...

def decode_string(data: str) -> str: ...

def decode_string_safe(data: str) -> str: ...
  • write_head将显式指明编码出的文本格式(utf16be),以便文本编辑器(如记事本)能够正确渲染,一般在写入文件时使用。

  • buf_rate指定读取文件的策略。当它为n时,则表示一次读取7n或者8n个字节。如果读到的字节长度小于预期,则说明长度不够, 此时,n将减半,恢复文件指针,重新读取。如果当n=1时长度仍然不够,就地encode/decode处理之。

  • encode_lendecode_len用于计算输出的长度

内部函数

  • 他们直接来自底层的C库,高性能,但是一般不需要在外部使用(除非是增加性能)
def _encode(data: BufferProtocol) -> bytes: ...

def _encode_safe(data: BufferProtocol) -> bytes: ...

def _decode(data: BufferProtocol) -> bytes: ...

def _decode_safe(data: BufferProtocol) -> bytes: ...

def _encode_into(data: BufferProtocol, dest: BufferProtocol) -> int: ...

def _encode_into_safe(data: BufferProtocol, dest: BufferProtocol) -> int: ...

def _decode_into(data: BufferProtocol, dest: BufferProtocol) -> int: ...

def _decode_into_safe(data: BufferProtocol, dest: BufferProtocol) -> int: ...

def is_64bits() -> bool: ...
  • _decode在解码b'='开头的数据时不安全解释器异常
  • _encode_into_decode_into直接操作缓冲区对象的底层指针,0拷贝,当然也和上面一样的问题,他们是没有检查的

✨ v0.3更新 ✨

融合了 CFFI 版本的成果,现在一个包可以同时在cpython和pypy上运行

本机编译

python -m pip install setuptools wheel cython cffi
git clone https://github.com/synodriver/pybase16384
cd pybase16384
git submodule update --init --recursive
python setup.py bdist_wheel --use-cython --use-cffi
  • 为了在windows上编译,需要加点料,把 这个 放进msvc的目录

后端选择

默认由py实现决定,在cpython上自动选择cython后端,在pypy上自动选择cffi后端,使用B14_USE_CFFI环境变量可以强制选择cffi