Camera Util
- Camera Util
Table of contents generated with markdown-toc
Overview
A collection of python threaded camera support routines for
- USB and laptop internal webcams
- RTSP streams
- MIPI CSI cameras (Raspberry Pi, Jetson Nano)
- FLIR blackfly (USB)
Also support to save as
- HD5
- tiff
- avi, mkv
Supported OS
- Windows
- MacOS
- Unix
The routines primarily use OpenCV or PySpin to interface with the camera. The image acquisition runs in a background thread to achieve maximal frame rate and minimal latency.
This work is based on efforts from Mark Omo and Craig Post.
Requirements
- PySpin for FLIR cameras
- opencv for USB, CSI cameras, RTSP network streams
- Windows uses cv2.CAP_MSMF
- Darwin uses cv2.CAP_AVFOUNDATION
- Linux uses cv2.CAP_V4L2
- Jetson Nano uses cv2.CAP_GSTREAMER
- RTSP uses cv2.CAP_GSTREAMER
On windows GSTREAMER is not enabled by default. If you want RTSP functionaly you need to custom built opencv. See my windows installation instructions on Github.
Installation
camera
cd "folder where you have this Readme.md file"
-
pip install .
orpython setup.py bdist_wheel
andpip3 install .\dist\*.whl
opencv
3. pip3 install opencv-contrib-python
tiff and hd5
https://www.lfd.uci.edu/~gohlke/pythonlibs/#imagecodecs
https://www.lfd.uci.edu/~gohlke/pythonlibs/#tifffile
https://www.lfd.uci.edu/~gohlke/pythonlibs/#h5py
Make sure the version matches your python installation (e.g. 3.8) and CPU architecture (e.g. 64).
blackfly
Spinnaker provides SDK and python bindings. The versions of those two programs need to match.
- Spinnaker SDK is at https://flir.app.boxcn.net/v/SpinnakerSDK, install development options
- download spinnaker_python from same location
To install the downloaded wheels, in CMD window:
-
cd Downloads
-
pip3 install imagecodecs....
-
pip3 install tifffile....
-
pip3 install h5py....
-
pip3 install spinnaker_python...
-
Make sure you have
C:\temp
directory if you use the example storage programs. -
To get better tiff performance, installing libtiff is advised: https://github.com/uutzinger/Windows_Install_Scripts/blob/master/Buildinglibtiff.md
To install OpenCV on Raspi:
cd ~
-
sudo pip3 install opencv-contrib-python==4.5.3.56
as time progresses the version numnber might need to be increased. sudo pip3 install tifffile h5py platform imagecodecs
How to create camera config files
A. Specifications of your camera
On Windows, the Camera utility will give you resolution options and frames per second.
To investigate other options you can use OBS studio (or any other capture program), establish camera capture device and inspect video options.
py -3 list_cv2CameraProperties.py
will show all camera options the video system offers. When an option states -1
it likely is not available for that camera.
B. Configuration file
Use one of the existing camera configutrations in examples/configs
or create your own.
As first step set appropriate resolution and frames per second.
As second step figure out the exposure and autoexposure settings.
Run Example
Run capture_display.py
from .\examples
You need to set the proper config file in the program. You should not need to edit python files in capture or streamer folder.
Example Programs
Display:
In general display should occur in main program.
OpenCV requires waitkey to be executed in order to update the display and limits update rate to about 50-90 fps.
Queue: Data transfer between the main program and capture and storage threads.
Examples:
-
capture_display.py
tests camera capture for all capture platforms except blackfly. -
blackfly_display.py
tests the blackfly capture module, displays images and reports framerate. -
capture_saveavi_display.py
display and save to avi files -
capture_savemkv_display.py
display and save to mkv files -
test_display.py
testing of opencv display framerate, no camera, just refresh rate. -
test_savehd5.py
testing of the disk throughput with hdf5, no camera -
test_savetiff.py
testing of the disk throughput with tiff, no camera -
test_saveavi.py
testing of the disk throughput with avi, no camera, only 3 color planes per image possible -
test_savemkv.py
testing of the disk throughput with mkv/mp4v, no camera, only 3 color planes per image possible -
test_blackfly.py
tests the blackfly capture module and reports framerate, no display -
blackfly_savehdf5.py
no display but incoporates saving to disk -
blackfly_savetiff.py
no display but incoporates saving to disk -
blackfly_savehdf5_display.py
display and incoporates saving to disk -
blackfly_savetiff_display.py
display andincoporates saving to disk
Pip upload
py -3 setup.py check
py -3 setup.py sdist
py -3 setup.py bdist_wheel
pip3 install dist/thenewpackage.whl
twine upload dist/*
Capture modules
blackflyCapture
Simplifies the settings needed for the Blackfly camera.
Supports trigger out during frame exposure and trigger in for frame start.
Optimized settings to achieve full frame rate with S BFS-U3-04S2M.
nanoCapture
Uses gstreamer pipeline for Jetson Nano.
Pipline for nvidia conversion and nvarguscamera capture.
Settings optimized for Sony IMX219 Raspi v2 Module.
cv2Capture
Uses the cv2 capture architecture.
The video subsystem is choosen based on the operating system.
rtspCapture
gstreamer based rtsp network stream capture for all platforms.
gstreamer is called through OpenCV.
By default OpenCV supports ffmpeg and not gstreamer. Jetson Nano does not support ffmpeg but opencv is prebuilt with gstreamer for that platform.
piCapture
Interface for picamera module. Depricated since cv2Capture is more efficient for the Raspberry Pi.
Changes
2022 - February added libcamera capture for Raspian Bullseye
2022 - January added queue as intialization option, updated cv2Capture
2021 - November moved queue into class
2021 - November added rtp server and client
2021 - November added mkvServer, wheel installation, cleanup
2021 - October added aviServer and multicamera example, PySpin trigger fix
2021 - September updated PySpin trigger out polarity setting
2020 - Release
Urs Utzinger
References
realpython:
https://realpython.com/python-concurrency/
https://realpython.com/python-sleep/
https://realpython.com/async-io-python/
python for the lab:
https://www.pythonforthelab.com/blog/handling-and-sharing-data-between-threads/
Example Camera Peformance
Sony IMX287 FLIR Blackfly S BFS-U3-04S2M
- 720x540 524fps
- auto_exposure off
OV5647 OmniVision RasPi
- auto_exposure 0: auto, 1:manual
- exposure in microseconds
- Max Resolution 2592x1944
- YU12, (YUYV, RGB3, JPEG, H264, YVYU, VYUY, UYVY, NV12, BGR3, YV12, NV21, BGR4)
- 320x240 90fps
- 640x480 90fps
- 1280x720 60fps
- 1920x1080 6.4fps
- 2592x1944 6.4fps
IMX219 Sony RasPi
- auto_exposure 0: auto, 1:manual
- exposure in microseconds
- Max Resolution 3280x2464
- YU12, (YUYV, RGB3, JPEG, H264, YVYU, VYUY, UYVY, NV12, BGR4)
- 320x240 90fps
- 640x480 90fps
- 1280x720 60fps
- 1920x1080 4.4fps
- 3280x2464 2.8fps
ELP USB Camera RasPi
- MJPG
- 320x240 and 640/480, 120fps
- auto_exposure, can not figure out out in MJPG mode
- auto_exposure = 0 -> static exposure
- exposure is about (exposure value / 10) in ms
- WB_TEMP 6500
Dell Internal USB
- 320x240, 30fps
- YUY2
- autoexposure ? 0.25, 0.74 -1. 0
- WB_TEMP 4600
- 1280x720, 30fps
- 620x480, 30fps
- 960x540, 30fps