Factor Graphs and the Sum-Product Algorithm

Presented by Dhananjay Ram Department of Electrical Engineering IIT Kanpur

Factor Graph
A way to represent probabilistic models Represented as a product of factors It is a bipartite graph with two types of nodes : Variable nodes and Factor nodes One node for each variable and each factor An edge connects a variable to a factor, if the variable appears in the factor

A simple example

Marginal Distribution

Sum Product Algorithm

At variable nodes : Take the product of all incoming messages except the message from the outgoing edge At factor nodes : Take the product of all incoming messages except from the destination with the internal factor and then sum it over all variables except the destination variable

Sum Product Algorithm

Initial messages : From a variable node, it is 1 From a factor node, it is the internal factor Normalization may be required at the variable nodes The probability of a variable is the product of all incoming messages

Sum Product Algorithm

Sum Product update rule

Single Marginal Function

All Marginal Functions

All Marginal Functions

Max Product Algorithm

One might be interested in valid configuration of largest a posteriori probability (APP), rather than the APPs for the individual symbols This case belongs to maximum likelihood sequence detection It can be achieved by replacing the sum operation with the max operation in sumproduct algorithm

Max Product Algorithm

The maximum value of a nonnegative realvalued function is viewed as the complete summary

Max Sum Algorithm

With the product of probability values there exists possibility of numerical underflow It is convenient to work with the logarithm of these numbers Logarithm can be used here because it is a monotonic function Logarithm simply has the effect of replacing the products with the sums

FACTOR GRAPHS WITH CYCLES

Cycles in the graph result in no natural termination of the sum product algorithm Exact inference cannot be obtained Different strategies can be made to converge the algorithm such as fixing the number of iterations or imposing some convergence criteria

Thank You