cthreading
cthreading implements Python 2 Lock, RLock, and Condition in C, speeding up threads synchronization and decreasing cpu usage.
Performance
cthreading eliminates the overhead of threading.RLock and threading.Condition which are implemented in pure Python in Python 2. In particular, threading.Condition is implemented using polling. In Python 3 threading.Condition is implemented without polling; cthreading implements a similar design in C.
$ time python whispers.py -m cthreading real 0m2.664s user 0m2.965s sys 0m0.808s $ time python3 whispers.py real 0m9.664s user 0m8.949s sys 0m1.812s $ time python whispers.py real 0m14.914s user 0m16.986s sys 0m12.690s $ time python whispers.py -m pthreading real 0m20.169s user 0m23.062s sys 0m17.022s
Your application is unlikely to have similar workload; do not expect this improvement.
For more info see https://github.com/nirs/cthreading/wiki/performance.
Usage
Import cthreading before any other module and monkeypatch the thread and threading modules. From this point, threading.Lock, threading.RLock, and threading.Condition are using cthreading.
import cthreading
cthreading.monkeypatch()
Note: cthreading will raise RuntimeError if the threading module was imported before cthreading.monkeypatch() is called.
Tested platforms
x86_64
- Fedora 22 / Python 2.7.10
- RHEL 7.2 / Python 2.7.5
- RHEL 7.2 / Python 2.7.10+ (upstream)
- RHEL 7.1 / Python 2.7.5
- RHEL 6.7 / Python-2.6.6
- Ubuntu 14.04 Server / Python 2.7.6
- Ubuntu 12.04 Server / Python 2.7.3 (python regression tests not available)
- Ubuntu 12.04 / Python 2.6.9 (Travis container)
- Ubuntu 12.04 / Python 2.7.9 (Travis container)
POWER8E
- RHEL 7.2 / Python 2.7.5
- RHEL 7.2 / Python 2.7.10+ (upstream)
Hacking
For rpm based distributions:
yum install python-devel python-test
For deb based distributions:
apt-get install python-dev libpython2.7-testsuite
Installing Python packages:
pip install pytest pytest-timeout yappi==0.93
Building and running the quick tests:
make
Before submitting patches, run the Python regression tests suite:
make regrtest
Check the Makefile for more info.
Similar projects
- pthreading - uses pthread_mutex and pthread_cond apis directly via ctypes. This introduces undefined behavior and actually slower and increases cpu usage in most cases compared to the original Python implementation.