Objectives_template

  Module 7: Motion Analysis

  Lecture 34: Mapping of motion into image plane, Aperture/occlusion/correspondence problems
 
 
 The Lecture Contains:
 
  Motion Analysis
 
  Aperture problem
 
Correspondence problem
 
  Occlusion problem
 
 
  
 
 

file:///D|/...0(Ganesh%20Rana)/MY%20COURSE_Ganesh%20Rana/Prof.%20Sumana%20Gupta/FINAL%20DVSP/lecture%2034/34_1.htm[12/31/2015 1:28:00 PM]

Objectives_template

  Module 7: Motion Analysis

  Lecture 34: Mapping of motion into image plane, Aperture/occlusion/correspondence problems
  
  Motion Analysis
 
Mapping of motion into the image plane
 
  In the pinhole camera model discussed earlier, the 3D world coordinates which are
  representing the distance of a point P from the focal point, are mapped by the central projection
 
equation into the image plane coordinates given by and .
 
  The speed of a mass point P in the 3D space is determined by the change of
  position P over time. From above equation, the movement observed in the image plane will then be
 
  of speed, :
 
Where, (7.1)

Substituting into, it becomes obvious that motions in the 3D world can

be invisible or be interpreted as not unique when observed in the camera image plane. For example,
no motion will be recognized if . In this case, motion occurs along one of the

rays of the projection towards the focal point. If temporal sampling of period T is applied (capturing a
sequence of image frames), the following relationship applies between the velocity vector and the
(spatially continuous, but time-discrete) displacement vector :
  (7.2)
Typically, velocity functions have a steady behavior over time due to mass inertia properties of
exterior-world objects. The speed is either constant or continuously accelerated .
The acceleration (second derivation of spatial coordinates over time) can only be analyzed if at least
three temporal positions of the point, to be connected by the motion trajectory, are known. In the
context of identification and unique interpretation of displacements in the image plane, the following
problems occur:
 
 

file:///D|/...0(Ganesh%20Rana)/MY%20COURSE_Ganesh%20Rana/Prof.%20Sumana%20Gupta/FINAL%20DVSP/lecture%2034/34_2.htm[12/31/2015 1:28:01 PM]

Objectives_template

  Module 7: Motion Analysis

  Lecture 34: Mapping of motion into image plane, Aperture/occlusion/correspondence problems
  
  Aperture problem:
 
If the window area for motion analysis only captures a part of an object which is homogeneous in
 
  amplitude, its motion cannot be determined uniquely, or eventually only along one direction (Fig.7.1).
  In general, it is at least necessary that a signal contains non-zero frequency components oriented by
  the direction where the estimation of motion shall be performed.
 
 
 
 
 
 

Fig 7.1
Correspondence problem:
In case of several equal objects or regular, periodic patterns, it may be impossible to identify the true
correspondence (Fig. 7.2). A unique correspondence may also be impossible to find in case of object
deformations (Fig. 7.3).
 

Fig 7.2
 
 

file:///D|/...0(Ganesh%20Rana)/MY%20COURSE_Ganesh%20Rana/Prof.%20Sumana%20Gupta/FINAL%20DVSP/lecture%2034/34_3.htm[12/31/2015 1:28:01 PM]

Objectives_template

  Module 7: Motion Analysis

  Lecture 34: Mapping of motion into image plane, Aperture/occlusion/correspondence problems
  
  Occlusion problem:
 
Due to the projection, parts of scenes (background or objects) can be occluded in one frame, but will
 
  become visible if the relative position between camera and object changes (Fig.7.4). If areas are
  covered or uncovered, they cannot uniquely be matched in motion estimation.
 
 
 
 
 
 
 

Fig 7.3
Considerations as made above, concerning relationships between speed and displacement, are
related to movements of mass points in the 3D space and the corresponding projected positions in
  the image plane. A point at position shall be subject to a translational displacement between
the sampling times of two frames. In most cases, the displacement v will not be constant over all
positions . However, the motion of coherent objects in the 3D space typically effects only
differential changes of local displacement shifts in the image. To investigate this effect, it is
necessary to establish a relationship between neighborhood displacement vector positions, which is
characterized by the displacement vector field and its changes . Fig. 7.5 shows examples
which could be described by a parameteric geometric transform, e.g. the affine transform (6.9),
 
 

file:///D|/...0(Ganesh%20Rana)/MY%20COURSE_Ganesh%20Rana/Prof.%20Sumana%20Gupta/FINAL%20DVSP/lecture%2034/34_4.htm[12/31/2015 1:28:01 PM]

Objectives_template

  Module 7: Motion Analysis

  Lecture 34: Mapping of motion into image plane, Aperture/occlusion/correspondence problems
  
 
 
 
 
 
 
 
 
 
 
 
  Fig 7.4

Fig 7.5
Generic descriptions of volumetric movements are well-known from continuum mechanics. Herein, a
rigid body is decomposed into an infinite number of an infinitesimal volume element
, and the movement within time dt is described by the differential changes of
these volume elements (Fig. 7.6). According to the fundamental theorems of kinematics, the
movement of infinitesimal volume elements can uniquely be described as a sum of translation,
rotation and deformation. Translation and rotation have 3 degrees of freedom each, while
deformation has another 6 degrees of freedom (each 3 for the changes in lengths of the sides of the
cube, 3 for the shear of the angles). The deformation of an infinitesimal volume element can fully be
interpreted as a linear transformation in a 3D coordinate system.
 
 

file:///D|/...0(Ganesh%20Rana)/MY%20COURSE_Ganesh%20Rana/Prof.%20Sumana%20Gupta/FINAL%20DVSP/lecture%2034/34_5.htm[12/31/2015 1:28:01 PM]

Objectives_template

  Module 7: Motion Analysis

  Lecture 34: Mapping of motion into image plane, Aperture/occlusion/correspondence problems
  
  By the projection onto the image plane, only the surfaces of objects become visible (except for the
  case of transparent objects). A very similar analysis can then be performed using infinitesimal 2D
 
area elements . The description of the displacement vector field can be described as
 
a sum of translation (2 degrees of freedom), rotation (one degree of freedom) and deformation (3
 
  degrees of freedom), which can be mapped into the 6 degrees of freedom supported by the affine
  transform (6.9). The change in the displacement vector field is then described as
  where A would be identical to the matrix from (6.9) when is at the center
  of the coordinate system. It is however clear from the earlier discussion that this model will not
  uniquely apply to larger, non-rigid objects of arbitrary shape; further, the occlusion problem is not
  considered at all. If occlusion occurs due to object motion, where parts of the background will
  become covered or uncovered, the assumption of continuity and differential changes of displacement
vector fields is no longer valid. In particular, differentiation will not be possible at the positions of
discontinuities. Fig. 7.6 shows possible configurations of displacement vector fields at the boundaries
between two regions which are moving in opposite directions (e.g. foreground/background). In Fig.
(7.7a,b) part of the background object is uncovered (the displacement vectors diverge at this
position), while in Fig. (7.7c,d) the case of covered background appears (the displacement vectors
  converge). It depends on the movement direction of the boundary, which of the two regions is the
foreground object. The width of the stripe which is covered or uncovered corresponds to the
displacement component which is perpendicular to the boundary. In the example shown, the newly-
uncovered or covered areas can be allocated either to region A (Fig. 7.7a,c) or region B (Fig.
7.7b,d).
 
 

file:///D|/...0(Ganesh%20Rana)/MY%20COURSE_Ganesh%20Rana/Prof.%20Sumana%20Gupta/FINAL%20DVSP/lecture%2034/34_6.htm[12/31/2015 1:28:02 PM]

Objectives_template

  Module 7: Motion Analysis

  Lecture 34: Mapping of motion into image plane, Aperture/occlusion/correspondence problems
  
 
 
 
 
 
 
 
 
 
 
 
 

Fig 7.7 (a,b,c,d)

The displacement vector field , which is still continuous in spatial coordinates, can be set in
relationship to the motion vector field , which is discrete in all three dimensions and
expresses the motion shift by units of pixel coordinates. In digital video processing, usually only the
motion vector field is directly processed, however it is necessary to understand its relationships with
the displacement vector field and the velocity vector field if the result of analysis shall be used for
semantic understanding and interpretation of the exterior-world motion. By substituting the following
equation into (7.2), also the direct relationship between the rally-continuous velocity vector u is give:
  (7.3)

The motion vector represents the translational motion at position , the

motion shift of one single pixel. D is the sampling matrix.

Motion feature extraction shall allow describing the behavior of an object or also camera motion by a
small set of motion parameters. According to a motion model selected (e.g. affine transform),
parameters must be estimated to fit the motion vector field over smaller or larger areas, or even for
the entire image plane if camera motion shall be estimated. In the following sections, the most
widely used classes of motion estimators, based on the optical flow principle and on matching
methods, are introduced. This is first done for the cases of translational motion parameter estimation;
the principles are then generalized to estimation of arbitrary non-translational motion parameters.
 
 

file:///D|/...0(Ganesh%20Rana)/MY%20COURSE_Ganesh%20Rana/Prof.%20Sumana%20Gupta/FINAL%20DVSP/lecture%2034/34_7.htm[12/31/2015 1:28:02 PM]