You are on page 1of 5


W . Bennett
Electrical Engineering Department Southern Methodist University Post Office Box 750338 Dallas, Texas 75275-0338 USA

Random field models have been successfully utilized in many applications requiring texture synthesis, classification, and segmentation. This class of models assumes each image pixel can be represented as a function of neighboring pixels and an additive noise sample. The effectiveness of these models is highly dependent on the choice of neighbor sets. Current approaches to selecting neighbor sets are based on ad-hoc methods. In this paper a systematic method which selects neighbor sets based on the correlation structure of texture images is presented and evaluated.

A number of random field (RF) models have been developed in the past. Among them are the simultaneous autoregressive (SAR) [ 11, conditional Markov (CM) [l], and long correlation (LC) [2] models. Each of these models require that a neighbor set of relative lattice positions be defined. For each neighbor location there is an associated neighbor coefficient as one of the model parameters. Modeling a specific image using a RF model thus becomes a problem of jointly estimating the neighbor set and the corresponding neighbor coefficients. Although methods for estimating the

coefficient values are well known, good techniques for selecting neighbor sets have not been previously available. Researchers have generally relied on ad-hoc trial and error approaches to select neighbor sets. Kashyap and Chellappa proposed a procedure for selecting among competing neighbor sets in [3], but provide no insight into how to identify the candidate neighbor sets. In practice, most applications employing RF models choose neighbor sets rather arbitrarily. This often results in neighbor sets which include more neighbors than are required to model the image effectively. Inclusion of these unnecessary neighbors add to the computational burden and can potentially degrade the performance of the model as an additional source of noise. In this work we address the difficulties of selecting RF neighbor sets by devising a scheme to select appropriate neighbors based on the correlation structure of the image under consideration. Although the effectiveness of our approach is demonstrated using the SAR RF model it is applicable to other spatial interaction RF models as well.

The correlation structure of an image to be modeled can be used to aid in the selection of appropriate neighbor sets. Neighbor positions which correspond to large correlations should


0-8186-6950-0194 $4.000 1994 IEEE

be favored over those with small correlations. This approach leads to neighbor sets which are tailored to the image under consideration. The number of neighbors that are selected in this manner can be determined by an iterative process where the effectiveness of the neighbor set is evaluated as each candidate neighbor is added to this set. The degree to which the model assumptions are satisfied can serve as this effectiveness measure.

Chellappa [3]. An image which obeys the S A R model can be reconstructed using the synthesis method detailed in [ 11, [4]. Selection of an appropriate neighborhood, N, is a challenging task.

2.2 SAR Model Neighborhood Estimation
The procedure presented here determines appropriate RF model neighbor sets based on the sample correlation function (SCF) [5] for an image defined on a torroidal lattice. The SCF is defied here as

2.1 SAR RF Model One of the most widely used RF models is the SAR model. For mathematical simplicity the SAR model is formulated using the torroidal lattice assumption. A location within an M x M two dimensional lattice is denoted by s = (i,j), with i j integers from the set J = { 0, 1, ..., M-1 }. The set of all lattice locations is denoted by Q = { s = (i,j): i j E J }. Let the image pixel value at location s be denoted by g(s). The mean of the image is then defined as E{g(s)} = a, Vs E Q. The mean subtracted image, denoted by y(s), is then y(s) = g(s)-a. The neighbor set of relative lattice positions required by the SAR model is defined as N = { r, = (k&J: m = 1, ..., p; k,l E J }. The SAR model relates each lattice position to its neighboring pixels according to the model equation

where y(-) is assumed to be a zero mean image and I represents a displacement (lag) in the spatial domain. Some properties of the SCF worth noting are: a) b) C) -1 I SCF(I) I 1 SCF(0,O) = 1 SCF(-I) = SCF(I)

The symmetry property c) results from the torroidal lattice assumption. The procedure for selecting neighbors is as follows, with p initially set to one:


where q(s) is independent identically distributed (i.i.d) random variables with zero mean and unit variance, p is the noise variance, er’sare coefficients which define the dependence of y(s) on its neighbor set N, and @ denotes modulo addition in each index. Model parameters p and 0 can be estimated using the approximate maximum likelihood technique developed by Kashyap and

Identify the p lattice displacements which have the largest corresponding SCF magnitudes for the texture image under consideration. In practice this search can be restricted to relatively small displacement values 1. The symmetry property of the SCF allows this search to be limited to displacements which lie within an asymmetric half-plane. The identified p lattice displacements are taken as the SAR neighborhood Np.



Using this Np estimate the corresponding SAR parameter vector Op.


Compute the residual image r(s) from the original image and the estimated SAR parameters.

The residual image can be viewed as an estimate of the SAR model noise source required to model the image with the specified model parameters. For an ideal N this would be a white noise image.

Due to the symmetry property of the SCF the selected neighbor sets are inherently unilateral, although this neighbor set can be viewed as the asymmetric half-plane of a symmetric neighborhood. Figure 1 graphs the maximum SCF obtained in step 4 of the procedure vs. number of neighbors for the herringbone texture image. From this graph it is apparent that the 5 selected neighbors provide the best decorrelation for this image, thereby making this the M which best satisfies the SAR model assumptions.
Herrin&one T e x t u e Residual Inoge


Evaluate the correlation structure of the residual image. The SCF of the residual image is the basis for measuring the effectiveness of the selected neighbor set. Determine the maximum SCF value of r(s) obtained for all displacement values. Compare this SCF magnitude to previously obtained values for this parameter. If it is less than values obtained during previous iterations the current Np does a better job of decorrelating the texture image. In this case record Np. Stop if the termination condition has been satisfied. For this study the iterations were terminated when subsequent iterations no longer produce an appreciable reduction in the maximum SCF value. Upon termination the last N p recorded in step 4 is taken as the selected neighbor set N . If the conditions in step 5 above are not satisfied, increment p and go to step 1. Note that in subsequent iterations N,, will include all lattice locations selected during prior iterations.






5 6 7 8 9 No. o f selected neiqi-bors

1 0 1 1


Figure 1. Image decorrelation vs. number of selected neighbors for the herringbone texture.


The effectiveness of the described method for neighbor set selection was evaluated by comparison of images synthesized using the selected neighbor set to the original texture image [6] from which the parameters were estimated. In addition, images were synthesized using a fixed-size neighborhood to judge the performance of this method relative to the traditional arbitrary selection techniques. Through a large number of synthesis experiments a 22 element unilateral neighbor set was determined to be the smallest



neighborhood which produced good results over a broad range of texture images. This neighborhood is shown in Figure 3. When compared to this fixed neighborhood approach the selected neighbor method required fewer neighbor elements to achieve good synthesis results. The improvement was most apparent for those textures which exhibit a strong directionality. Example images are shown in Figure 2. The images synthesized from the

selected neighbor sets are shown in the center row with the number of selected neighbors given beneath each image. The selected neighbor sets for each of the sample textures are shown in Figure 3. Although in some cases the images synthesized using our neighbor selection method are visually similar to those obtained using the fixed neighbor set our approach requires fewer neighbors and is based- on a systematic selection method.


5 neighbors

2 neighbors

1 neighbor

17 neighbors

Figure 2. Original images are shown in the topmost row. Images synthesized from the neighbor sets selected by our approach are shown in the center row. The bottom row shows the same images synthesized from a 22 element fixed neighbor set. This neighbor set was empirically determined to be the smallest fixed neighbor set which consistently produced good results.


Figure 3. SAR model neighbor sets f o left to right: 22 element fixed neighbor set, herringbone texture selected rm neighborhood, grass texture selected neighborhood, wood texture selected neighborhood, and wool texture selected neighborhood.

We have presented a new method for selection of neighbor sets in F models. W Through image synthesis we have compared the performance of the models using the selected neighbors with results obtained using a larger, fixed-size neighborhood. Although presented as a method for selecting neighbors for use in image synthesis, this approach could be easily adapted to other RF model applications such as classification problems. Also, since the approach is independent of the RF model details it is not limited to a specific type of model such as the SAR model but is instead applicable to any spatial interaction RF model. Future work will investigate the performance of this approach in multi-class applications such as texture classification and segmentation.


R. L. Kashyap and P. M. Lapsa, "Synthesis and Estimation of Random Fields Using Long Correlation Models", IEEE Trans. on Pattern Anal. Machine Intell., vol. PAMI-6, No. 6, pp. 800809, Nov. 1984.


R. L. Kashyap and R. Chellappa, "Estimation and Choice of Neighbors in Spatial Interaction Models of Images", IEEE Trans. Inform. Theory, vol. IT-29, Jan. 1983.
A. Khotanzad and R. L. Kashyap, "Feature Selection for Texture Recognition Based on Image Synthesis", IEEE Trans. on Sys., Man, and Cyber., vol. SMC-17, NO. 6, pp. 298-303, Nov./Dec. 1987. R. Chellappa and R. L. Kashyap, "On the Correlation Structure of Random Field Models of Images and textures", Proc. of 6th Int. Con. on Pattern Recognition, Dallas, Tx, Aug. 1981.




R. Chellappa and R. L. Kashyap, "Synthetic Generation and Estimation in Random Field Models of Images", Proc. of 6th Int. Conf. on Pattern Recognition, Dallas, Tx, Aug. 1981.


P. Brodatz, Textures - a Photographic Album for Artists and Designers. Dover, New York (1966).