scannertools: video processing toolkit  

Scannertools is a Python library of easy-to-use, off-the-shelf pipelines written using the Scanner video processing engine. Scannertools provides implementations of:

• Object detection
• Face detection
• Face embedding
• Gender detection
• Pose detection
• Clothing detection
• Optical flow
• Shot detection
• Random frame access

See the documentation for details.

Usage

Here's an example using scannertools to extract faces in every 10th frame of a video, and then to draw the bounding boxes on one of those frames.

from scannertools import face_detection, Video, imwrite
import scannerpy
import cv2

# Get a reference to the video
video = Video('path/to/your/video.mp4')
frame_nums = list(range(0, video.num_frames(), 10))

# Run the face detection algorithm
db = scannerpy.Database()
face_bboxes = face_detection.detect_faces(db, videos=[video], frames=[frame_nums])

# Draw the bounding boxes
frame = video.frame(frame_nums[3])
for bbox in list(face_bboxes.load())[3]:
    cv2.rectangle(
        frame,
        (int(bbox.x1 * video.width()), int(bbox.y1 * video.height())),
        (int(bbox.x2 * video.width()), int(bbox.y2 * video.height())),
        (255, 0, 0),
        4)

# Save the image to disk
imwrite('example.jpg', frame)

For more examples, see the examples directory. For the API reference, see our documentation.

Installation

Scannertools requires the Python packages for our three libraries Scanner, Storehouse, and Hwang to be installed. Scannertools also has optional dependencies for certain pipelines, e.g. TensorFlow, OpenCV, and so on.

Docker

We recommend using our prebuilt Docker images for Scannertools (scannerresearch/scannertools). For example:

will@scannertools ~ ❯❯❯ docker run -ti scannerresearch/scannertools:cpu-latest bash
root@7aec5a7d9198:/app# python3 /opt/scannertools/examples/face_detection.py
...
Wrote video with objects drawn to /app/sample_faces.mp4
root@7aec5a7d9198:/app# exit
will@scannertools ~ ❯❯❯ ls
sample_faces.mp4

If you want to use our GPU images, then you need to install nvidia-docker. Then run Docker with the --runtime nvidia flag, e.g.

will@scannertools ~ ❯❯❯ docker run --runtime nvidia \
    -ti scannerresearch/scannertools:gpu-9.0-cudnn7-latest bash

Native

To install from scratch, first follow the installation instructions in the Scanner repository to get Scanner, Storehouse, and Hwang. Then run:

pip install scannertools

You will need to download dependencies for different pipelines. Currently you need:

• Face detection and face recognition: facenet
• Gender detection: rude-carnie
• Clothing detection: pip install torch==0.3.1 torchvision

See the Dockerfile for how to set up the PYTHONPATH appropriately for these dependencies.