Videochef is a Python library that makes it easier to work with videos in scientific settings. It builds on the low-level ffmpeg and pyav libraries to create a fast, Pythonic way to read and write videos.

Features include:

• VideoReader and VideoWriter classes that are fast and "just work"
• Batch processing of embarassingly parallel analyses (requires multiple CPUs)
• Easy creation of peri-event video galleries
• Precise frame counting in VideoReader makes it easy to align videos across multiple cameras that may have sparse dropped frames or slightly differing start and end points
• ...your contributed feature here!...

When should you use videochef vs. other python video libraries?

Install

pip install videochef, or clone the repo and pip install -e . from inside the repo. Don't forget to use a conda env or a venv!

Examples

Easily read only frames that match across streams:

matched_frames = [[0,2,4], [1,3,5]]  # say the second camera started 1 frame early, and each camera dropped a frame.
with VideoReader('/path/to/vid0.avi', frame_ixs=matched_frames[0],) as bottom_vid, \
    VideoReader('/path/to/vid1.avi', frame_ixs=matched_frames[1],) as top_vid:
    for i, (top_frame, bottom_frame) in enumerate(zip(top_vid, bottom_vid)):
        # do some analysis on the matched frames

Make peri-stimulus video galleries:

stim_frames = [6000, 8020, 12100, 15001]  # some recurring stimulus or event
fps = 30
window = (-1,1)  # seconds
peri_evt_frames_list = [np.arange(fr + window[0]*fps, fr + window[1]*fps) for fr in stim_frames]

videochef.viz.peri_event_vid(
    '/path/to/vid.avi',
    '/path/to/peristim_vid.avi',
    peri_evt_frames_list=peri_evt_frames_list,
    event_frame_num_in_vid=(0 - window[0])*fps,  # event onset will be marked in the corner
    out_fps=fps/2,
)

Do some complex analysis on every other frame of a video, in parallel (requires multiple CPUs):

def my_complex_analysis(frame):
    """Takes one video frame and analyzes it, returning an annotated frame and some scalars.

    Returns:
        A tuple!
    """
    # ... processing ...
    processed_frame = frame + 1  # dumb example
    scalar_results_dict = dict(attr1='foo', attr2='bar')
    return (processed_frame, scalar_results_dict)

step = 2
videochef.chef.video_chef(
    my_complex_analysis,
    '/path/to/my/vid.avi',
    output_types=['video', 'arrays'],  # arrays will be saved as npz's
    max_workers=3,  # ncpus - 1
    every_nth_frame=step,
    proc_suffix='_complexly_analyzed',
)

Authors

Caleb Weinreb wrote the core ffmpeg code, the initial reader/writer classes, and the peri-event gallery code. Jonah Pearl wrote the parallel processing module, updated the reader class to be more efficient, and updated the writer class to "just work" with color videos.

Roadmap

• properly benchmark!
• figure out GPU encoding for writer

Tested and works well with:

• AVI files, encoded with ffv1, and pixel format gray8
• MP4 files, encoded with h264, and pixel format yuvj420
• Likely works well with most grayscale AVIs and color/gray MP4s. Other formats tbd.

Citations

Logo adapted from freepik