Real-time Discriminative Background Subtraction

Brief Description

We examine the problem of segmenting foreground objects in live video when background scene textures change over time.
We formulate background subtraction as minimizing a penalized instantaneous risk functional--- yielding a local on-line discriminative algorithm that can quickly adapt to temporal changes. We analyze the algorithm's convergence, discuss its robustness to non-stationarity, and provide an efficient non-linear extension via sparse kernels. To accommodate interactions among neighboring pixels, a global algorithm is then derived that explicitly distinguishes objects versus background via maximum a posteriori inference in a Markov random field (implemented via graph cuts). By exploiting the parallel nature of the proposed algorithms, we develop an implementation that can run efficiently on parallel graphics processors, suitable for real-time video analysis >=75 fps on a mid-range GPU.

n

From left to right:: three typical scenarios that are characterized by these non-stationary background distributions: drifting, jumping and multi-modal switching.

We explicitly connect the problem of background subtraction to work in on-line learning and novelty detection, which possess a rich literature and well-studied theoretical principles. In addition, we present a series discriminative on-line learning algorithms based on kernels (called SILK and SILK-GC) that provide a principled method for modeling the spatial-temporal
characteristics of the background subtraction problem. our algorithm is designed to work with GPUs, yielding a processing speed of 60-120 frames per second (FPS).

SILK, the proposed sparse on-line learning paradigm

Results: (readme)

• SILK on two sequences
  • rock
  • traffic
• SILK and SILK-GC on five sequences
  • beach
  • trees
  • lights
  • jug
  • railway

Code:

• SILK for CPU code is here
• SILK for GPU code is here
• SILK-GC for GPU code is here

Results

Rock Video Sequence (right: SILK result)

Traffic Video Sequence (right: SILK result) , key frames

SILK/SILK-GC on Five Video Sequences

	beach	Tree	Lights	Jug	Railway
an image
curr image
Grd. Truth
MoG
SILK
SILK-GC
SILK-GC

Reference

Li Cheng, M. Gong, D. Schuurmans, and T. Caelli. Real-time Discriminative Background Subtraction. IEEE Trans. Image Processing, 20(5), 1401-1414, 2011.