GPU Detection

GPU-accelerated object detection can be enabled as long as you have a compatible GPU (CUDA-enabled GPUs).

If you have a compatible GPU, make sure the following required libraries are installed and configured for use.

CUDA Toolkit 11.8
cuDNN for CUDA 11.x

Windows installation

Linux installation

Configure hardware-accelerated GPU detection

To enable hardware-accelerated GPU detection on a device:

Go to the "Edit Device" page.
Enable "Show Advanced Settings."
Go to the "Monitor Settings" section.
Enable "Use GPU for detection."

At this point, StreamShuttle will try to use the GPU for object detection. It will fall back to the CPU if the required GPU libraries are not found. Verify that it is using the GPU by monitoring the logs.
When enabling GPU detection for the first time, it's usually best to have only one device running. This makes it easier to find relevant information in the logs.

Monitor the logs

On Linux: Open the terminal and run the following command.

tail -f ~/.config/StreamShuttle/streamshuttle/pm2/logs/hub-1-out.log

On Windows: Open PowerShell and run the following command.

Get-Content $env:USERPROFILE\AppData\Roaming\StreamShuttle\streamshuttle\pm2\logs\hub-1-out.log -Wait -Tail 30

Every time a device stream is restarted, information from the object detector will be logged. Seeing the following line confirms that the GPU is being used:

error: tensor worker stderr USING GPU: YES

If the GPU detector is not working, you will instead see the following:

error: tensor worker stderr USING GPU: NO

If the USING_GPU option is not working, the preceding lines will generally explain why. Look for any line that begins with the following:

error: tensor worker stderr

Configure GPU detection for a device

After GPU detection is running, you can increase the object detection rate.

Go to the "Edit Device" page.
Enable "Show Advanced Settings."
Go to the "Monitor Settings" section.
Increase "Object detection FPS."
- Experiment to find what works best for your configuration. You probably will not see much benefit from a value greater than 5.

Use the global detector

Depending on how many devices you have, it may make more sense to use a "global" detector rather than running a detector for each device. This will decrease the total amount of graphics card memory used.

To use the global detector, enable "Run global object detector" in the "StreamShuttle Control Panel."

Hub settings with Run global object detector enabled

Once it's enabled in the control panel, you can enable the device-specific option.

Go to the "Edit Device" page.
Enable "Show Advanced Settings."
Go to the "Monitor Settings" section.
Enable the "Use the global detector" option.

Depending on your specific usage, such as the number of cameras and type of GPU, it may make sense to add hubs and create more "Global Detectors." This lets you spread the load as needed.

For example, each detector requires roughly 600MB of GPU memory. If you have a smaller GPU, running 8 separate detectors might exceed the available GPU memory. Instead, you could create a global detector and have all devices use it. This would occupy only 600MB instead of 4800MB for a detector on each device.

Continuing with the same example, the FPS capacity may become a bottleneck if a "Global Detector" handles more than 8 devices. In that case, add another "Global Detector." This uses only 1200MB of memory while handling roughly twice the FPS.

The last scenario is to use multiple GPUs. You can assign the GPUs available to each hub through environment variables.