VisionWorks Toolkit Reference

December 18, 2015 | 1.2 Release

 All Data Structures Namespaces Files Functions Variables Typedefs Enumerations Enumerator Macros Groups Pages
IME : Refine Motion Field

Detailed Description

Iteratively refines motion field by applying the motion vectors to a center block in a 3x3 block neighborhood.

In this selection, the algorithm uses 14 motion vectors from the surrounding blocks, excluding second best vectors from top left and bottom right corner blocks, and 2 motion vectors from center block. The figure below illustrates the refinement stage iteration for a single block \( (i,j) \):

refine_motion_field.png

Best motion vectors for the central block is chosen by using a cost formula which takes in the SAD value of the block with this particular motion vector and its Manhattan distance to motion vectors in the 3x3 block neighborhood. The cost formula used in assessing the best two motion vectors is as follows:

\[ cost(MV_{cur}) = SAD(MV_{cur}) + iter * smoothnessFactor * \sum_{MV} dist(MV_{cur}, MV) \]

Local smoothness in the cost equation is the sum of manhattan distance values to all the motion vectors to the tested motion vector if it was to be picked as the representative for the current block.

After going over 16 motion vectors in 3x3 neighborhood, the best 2 motion vectors are kept as representative motion vectors for the current block, so that

\[ dist(MV0, MV1) >= mvDivFactor \]

This operation is repeated a few iterations since in each iteration, the motion vectors from the neighboring blocks are refined too, producing a smoother motion field.

Weight of the smoothness is increased in each iteration with the assumption that the quality of the motion vectors are improved.

Note
The primitive uses pre-calculated SAD tables, which can be obtained from the IME : Create Motion Field primitive.

Functions

vx_node nvxRefineMotionFieldNode (vx_graph graph, vx_image in_mv0, vx_image in_mv1, vx_image sad_table, vx_image out_mv0, vx_image out_mv1, vx_int32 searchWindowWidth, vx_int32 searchWindowHeight, vx_int32 numIterations, vx_float32 smoothnessFactor, vx_int32 mvDivFactor)
 [Graph] Iteratively refines motion field by applying the motion vectors to a center block in a 3x3 block neighborhood. More...
 
vx_status nvxuRefineMotionField (vx_context context, vx_image in_mv0, vx_image in_mv1, vx_image sad_table, vx_image out_mv0, vx_image out_mv1, vx_int32 searchWindowWidth, vx_int32 searchWindowHeight, vx_int32 numIterations, vx_float32 smoothnessFactor, vx_int32 mvDivFactor)
 [Immediate] Iteratively refines motion field by applying the motion vectors to a center block in a 3x3 block neighborhood. More...
 

Function Documentation

vx_node nvxRefineMotionFieldNode ( vx_graph  graph,
vx_image  in_mv0,
vx_image  in_mv1,
vx_image  sad_table,
vx_image  out_mv0,
vx_image  out_mv1,
vx_int32  searchWindowWidth,
vx_int32  searchWindowHeight,
vx_int32  numIterations,
vx_float32  smoothnessFactor,
vx_int32  mvDivFactor 
)

[Graph] Iteratively refines motion field by applying the motion vectors to a center block in a 3x3 block neighborhood.

Parameters
[in]graphSpecifies the graph.
[in]in_mv0Specifies the input motion field (first motion vectors). The motion vectors are assumed to be stored in Q14.2 format (NVX_DF_IMAGE_2S16).
[in]in_mv1Specifies the input motion field (second motion vectors). It must have the same size and format as in_mv0 image.
[in]sad_tableSpecifies the input image with SAD table. It must have \( [W * SW * SH, H] \) size, where \( W \times H \) - in_mv0 size and \( SW \times SH \) - search window size. The values are assumed to be stored in U32 format (VX_DF_IMAGE_U32). The SAD table can be obtained via IME : Create Motion Field primitive.
[out]out_mv0Specifies the output motion field (first motion vectors). It must have the same size and format as in_mv0 image.
[out]out_mv1Specifies the output motion field (second motion vectors). It must have the same size and format as in_mv0 image.
[in]searchWindowWidthSpecifies the search window width.
[in]searchWindowHeightSpecifies the search window height.
[in]numIterationsSpecifies the number of iterations.
[in]smoothnessFactorSpecifies the smoothness factor in cost formula.
[in]mvDivFactorSpecifies the best motion vectors diversity factor.
Returns
A valid node reference or an error object (use vxGetStatus).
vx_status nvxuRefineMotionField ( vx_context  context,
vx_image  in_mv0,
vx_image  in_mv1,
vx_image  sad_table,
vx_image  out_mv0,
vx_image  out_mv1,
vx_int32  searchWindowWidth,
vx_int32  searchWindowHeight,
vx_int32  numIterations,
vx_float32  smoothnessFactor,
vx_int32  mvDivFactor 
)

[Immediate] Iteratively refines motion field by applying the motion vectors to a center block in a 3x3 block neighborhood.

Parameters
[in]contextSpecifies the context.
[in]in_mv0Specifies the input motion field (first motion vectors). The motion vectors are assumed to be stored in Q14.2 format (NVX_DF_IMAGE_2S16).
[in]in_mv1Specifies the input motion field (second motion vectors). It must have the same size and format as in_mv0 image.
[in]sad_tableSpecifies the input image with SAD table. It must have \( [W * SW * SH, H] \) size, where \( W \times H \) - in_mv0 size and \( SW \times SH \) - search window size. The values are assumed to be stored in U32 format (VX_DF_IMAGE_U32). The SAD table can be obtained via IME : Create Motion Field primitive.
[out]out_mv0Specifies the output motion field (first motion vectors). It must have the same size and format as in_mv0 image.
[out]out_mv1Specifies the output motion field (second motion vectors). It must have the same size and format as in_mv0 image.
[in]searchWindowWidthSpecifies the search window width.
[in]searchWindowHeightSpecifies the search window height.
[in]numIterationsSpecifies the number of iterations.
[in]smoothnessFactorSpecifies the smoothness factor in cost formula.
[in]mvDivFactorSpecifies the best motion vectors diversity factor.
Returns
A vx_status enumerator.
Return values
VX_SUCCESSNo errors.
VX_ERROR_INVALID_REFERENCESupplied parameters are not a valid references.
VX_ERROR_INVALID_PARAMETERSSupplied parameters are not valid.
VX_ERROR_INVALID_SCOPESupplied parameters are virtual objects that cannot be used in immediate mode.
VX_FAILUREInternal error in primitive implementation, check log for detailed information (Framework: Log).