Pattern recognition by optical moments

Pattern recognition by optical moments

Volume 20, number 1 OPTICS COMMUNICATIONS January 1977 PATTERN RECOGNITION BY OPTICAL MOMENTS A.F. FERCHER University of Essen, Unionstrasse 2, D...

263KB Sizes 0 Downloads 52 Views

Volume 20, number 1


January 1977


University of Essen, Unionstrasse 2, D.4300 Essen, Fed. Rep. Germany Received 30 September 1976 A new method is described for pattern recognition employing moments derived from the diffraction pattern of the object

1. Introduction Optical pattern recognition can be done in object space or in Fourier space. Because of the reversibility of the Fourier transformation both approaches are equivalent, at least in principle. This paper presents a new method for optical pattern recognition that employs the derivatives of the intensity of the Fraunhofer diffraction pattern for feature extraction. The advantages of this procedure are: (1) the intensity of the diffraction pattern is shift invariant, (2) the diffraction pattern is centered at the optical axis, and (3) the procedure responds to phase structures, e.g., in the case of biological cells. We represent the object by a two-dimensional function O(x, y), which describes the complex light amplitude distribution in the plane immediately behind the object, assumed to be illuminated by a plane wave travelling in the z-direction. The light distribution in the diffraction pattern, U(u, v), is the Fourier transform of O(x, y), and the intensity I(u,v) is the square of the modulus of U(u, u). Of course, in going from U(u, v) to I(u, v) a loss of information occurs: the phases of the Fourier transform of O(x, y) are discarded. This is equivalent to the interpretation that feature extraction from I(u, v) corresponds to the extraction of properties from the autocorrelation function +¢o

c(x,y) = f O(s, t) o*(x-s, y - t ) dsdt, the asterisk denoting the complex conjugate. In the paper we are concerned with pattern recognition by optical moments, where the optical moment

of order m, n is defined in terms of Riemann integrals by 4-oo

M(m, n) = f



xmy n c(x, y) dxdy;

m , n = 0 , 1,2 ....


As outlined in section 2, the optical moments perform a very useful kind of information reduction, in that the lowest moments give a rough estimate ofc(x,y), and increasing orders of moments give an increasing definition of the autocorrelation. These measures meet the general requirement that feature vectors be invariant with respect to certain pattern variations and that they have sufficient separability in the feature vector space.

2. Optical moments In the theory of statistics the second limit theorem [1 ] states that if all moments of a density function are bounded the density function is uniquely determined by its moments. We expect that two functions with a certain number of moments in common will resemble each other to some extent. In fact, it can be shown that, if two arbitrary functions are continuous in the range a to b and have equal moments up to order K, then they must have the same least squares approximation by a uniformly convergent series of polynomials of degree K. Let the functions be f t ( x ) and f2(x) and the approximation be the finite series of powers Y~K=0 an x n. The principle of least squares ap81

Volume 20, number l


proximation demands that b , K 2

inversely proportional to what may be called the "structure content". The total scattered energy flux is proportional to the "energy" of the object :


be minimum. From this it follows that the coefficients a n are determined by the moments m i of the functions f/(x): b b K


f I(u, v) dudv





=fl 0


January 1977

f O(x,y) O• (x,y) dxdy.

As long as the f)(x) have the same moments, the approximation will be the same. The ruth differential coefficient with respect to u of the intensity I(u, o) of the Fraunhofer diffraction pattern is

This may be a definition of the energy of a two-dimen. sional object function. The structure content as defined above will depend on the object energy. A more reliable figure for the structure content is the equivalent width taking into account the object energy:

d " / ( u , v) _ (2rri) m du m

f +~ f +_~c(x, y) dxdy

X f



x'nc(x,y) e2rri(ux+v.V)dxdy.


(An analogous expression holds for dml(u, u)/du m.) Calculating this derivative at the origin u =v=0, we obtain (2Tri) m+n times the optica] moment M(m,n):


a"'+~l(u, v)

- - -

M(m, n) = (2rri) m+" ournoon u=0, v=0


To calculate the optical moments of a pattern the derivatives of the intensity at the optical axis must be determined. From a practical point of view it is more useful to determine the one-dimensional moments M(m, 0) and M(O, n) independently. This is possible if some kind of pre-orientation of the pattern can be achieved, as is indicated in section 3.

2.1. Zero moments M(0,O) = I(O,O)

= f c(x,y) dxdy

expresses the fact that the definite integral of a function taken from ,,o to +~o is equal to the value of its transform at the origin. On the other hand the autocorrelation function c(x, y) has its maximum at the origin and decreases monotonically (except in case o f periodic structures) with increasing x and y, Increasing spatial frequency content of O(x, y) will emphasize the decrease of c(x,y). Accordingly, M(0,0) is 82


f +_~f+~ l(u, o) dudv"

Weak scattering biological cells can also be represented by two-dimensional functions O(x, y). The function O(x, y) again describes the complex light amplitude behind the object illuminated by a plane wave travelling in the z-direction. It has been shown [2] that O(x,y) is given by the complex function exp [i¢~(x, y)], where D

(a(x,y)=2~ f [~(x,y,z)--rlO] dz, o rl(x, y, z) being the refractive index of the object, rl0 is the refractive index of the surrounding medium, D the object extension in the z-direction. Weak scattering biological cells can be treated as aqueous solutions of proteins and their refractive index is given by r/= r20 + aC, where r~0 is the refractive index of the solvent (water), C is the concentration and a is the specific refraction increment of proteins. If the object field is larger than the object itself, as is the case in pattern recognition procedures, O(x, y) represents the difference of the object field including the object minus the empty object field: O(x, y) = U0 (exp [iqs(x,y)] - 1}, with ~(x, y) = (2rr/X) fg aC(x, y, z) dz. g 0 is the illuminating plane wave travelling in the z-direction. Now the total scattered energy flux is

Volume 20, number 1

f f _oo


O(x,y) O*(x,y) dxdy = 2 U 2


_~ ,m




i ~


As long as D

7r _2~ f ~C(x, y,z)dz <3~ , o eq. (4) gives a linear dependence ofO(x,y) O*(x,y) on fg eeC(x,y, z) dz, i.e., the total scattered energy flux is proportional to the dry matter content of the cell.

Because the intensity l(u, v) is a real physical quantity, the autocorrelation function c(x, y) is hermitian. As a consequence, even moments describe the character of the real part of c(x, y)and odd moments describe the imaginary part. So fas as real object functions O(x, y) are concerned, the autocorrelation function is a real even function, and all odd moments vanish. The following treatment for the weak scattering biological cell will be limited to one dimension. To make the infinite correlation integral exist we must represent the object by a function that is zero outside a i a ~ < x _--~ < l~- a, The light distribufinite object field - ~tion of a cell in a bright field is thus

O(x) = rect (x/a) {exp [ig(x)] - 1 ) , with ~(x) = (27r/X) e~fg C(x, z) dz, fg C(x, z) dz being the integrated protein concentration and rect

m {' =



A straightforward computation gives the autocorrelation of this function: +~

c(x)= f

rect ( u ) f e e t ( ~ f )

x (1 + cos [ ~ ( u ) - ~ ( u - x ) ] - 2 cos ~(u)) du +i

The argument o f both real and imaginary parts of constant factor (27r/X) a] the difference between the protein concentration integrated in the z-direction, q~(u), and a duplicate of q~(u) shifted by x. The first optical moments M(1,0) and M(0,1) of the object O(x, y) are, according to eq. (3), 1/(2hi) times the corresponding slope of the intensity in the diffraction pattern at the optical axis. We define a mean abscissa (x) such that the area o f c ( x , 0) times (x) is equal to the first moment [3]. Then x indicates where the imaginary part o f c(x, 0) is mainly concentrated. From eq. (2) follows

c(x) is [neglecting the

(x) =

2.2. Highermoments


_f rect(U) rect(~f-) sin[c~(u)-O(u-x)]du"

January 1977


f+==xe(x,0) dx _ M(1,0) f+Soe(x, O)dx M(0,0) 1

0 I(u, v)

2rril(0, 0)



An analogous expression holds for (y). The second optical moments o f a pattern are defined as 1/(4rr 2) times the curvature of I(0,0). It can be applied to define the mean-square abscissas (x 2 ) and (y2) o f the real part of the autocorrelation function, e.g. _M(2,0) _ (x2) - M ( 0 , 0 )

1 4rr21(0, 0)

O21(u,v) (0,0) ~u 2


and an analogous expression for (y2). These quantities are quite graphic; in dynamics the square of the radius of gyration of a rod with a mass density o f c ( x , 0) or c ( 0 , y ) is the same a s (x 2) or (y2), respectively. The third optical moments M(3,0) and M(0, 3) may be used to measure the skewness or degree of asymmetry o f the imaginary part o f c (x, y). Comparing skewness and mean abscissa we see that skewness dominates the more the extension o f c(x, y) increases. The fourth optical moments M(4, 0) and M(0, 4) can be applied to measure the degree of peakedness of the kurtosis o f the real part of c(x, y). Several other graphic measures can be derived from the moments so far described and from higher moments. The crucial question is, however, to what precision can the intensity distribution l(u, v) in the diffraction pattern be determined? As we must compute the derivatives of l(u, v), this is the central question. In the subsequent section a new and simple method for high precision intensity measurements in diffraction patterns is presented. 83

Volume 20, number 1


3. Measurement of the intensity distribution

l(u, u)

The method described here is based on the Doppler effect. If the scattering object, e.g., the biological cell moves, the scattered light will be Doppler-shifted. By superposition of the scattered light with a coherent background the Doppler frequency appears as a beat frequency of the intensity. This beat frequency forms the basis for a phase sensitive detection and amplification and thus allows for precise intensity measurements in the diffraction pattern. In fig. 1 the situation in a flow-through system for recognition of biological cells is depicted. The ractangular cross section of the channel causes some preorientation. The cell moves with a velocity V. The object field is described as above by rect

(x/a) rect (y/a) exp


The diffraction pattern in the (u, u)-plane consists of two parts: a very strong narrow peak caused by the bright field rect (x/a) rect (y/a) and a comparably broad pattern caused by the object exp [[email protected](x, y)] (the object is much smaller than a). The bright peak and also secondary scattering of the bright field illumination can be viewed as a noise background added to the intensity distribution in the diffraction pattern of the object. If the object moves and a coherent background light is added, the intensity distribution of the object diffraction pattern will be separable from all additional intensities as it is only the object light that is Doppler shifted. The Doppler frequency o f the scattered light is Cb





I.t D-TO

1o-2o Fig. 2. Positions of the detectors Dij for intensity measurements in the diffraction pattern [= (u, o)-plane I. At the optical axis there is no Doppler shift. The total number of periods N of the sinusoidal intensity variation at position on the u-axis when the object moves across the field is N = (2a/X) sin ½0 cos½0.


If several detectors are positioned along the u-axis, as indicated by Dio, i = -+1, -+2, ... in fig. 2, each of these will register a fixed number N of periods proportional to 0 as given by eq. (6). To determine the optical moments along the y-axis, measurements should be made on the o-axis in the diffraction pattern. To utilize the Doppler frequency, here too the measurements will have to be made at positions D0i, i=+-1, +2 .... a small distance from the u-axis. In the case indicated in fig. 2, all signals at the Doi detectors will have the same frequency. Phase sensitive detection of all signals is possible, as their frequencies are connected by 0. Any one of the signals can act as a reference or gating signal.

(2 V/X) sin½0 cos½0..

To determine optical moments up to the nth order we have to measure n + 1 intensity values near the ontical axis. X


V ~,.y


Fig. 1. A flow-through system for recognition of biological cells by optical moments. Coherent background in (u, o)-plane not indicated. 84

January 1977

4. Numerical examples In this preliminary study only zero moments are considered. The exact Mie theory is applied to calculate the diffraction patterns of homogeneous and of coated spheres. Fig. 3 shows the total energy scattered per unit volume of the cell versus cell diameter. Three graphs are shown: two homogeneous spheres of relative refractive indices m = r/it/0 = 1.01 and 1.05 and one coated sphere with refractive indices of 1.05 in the core and 1.025 in the coat. In the latter case the core diameter equals half the cell diameter. The three graphs show the functional dependence as described by eq. (4): the diffracted energy flux per unit volume

Volume 20, number 1


January 1977












cell diameter

Fig. 3. Total energy scattered per unit volume of the cell versus the cell diameter. Graph a: homogeneous sphere, relative refractive index m = 1.01. Graph b: homogeneous sphere, relative refractive index m = 1.05. Graph c: coated sphere, m = 1.05 in the core and m = 1.025 in the coat. is not constant but increases in case o f a h o m o g e n e o u s refractive i n d e x according to




equivolent w/dth [erbitc unfs]

Fig. 4. Cell diameter versus equivalent width. Graphs a, b and c represent the same models as in fig. 3. Acknowledgement

i.e., the total diffracted energy flux is p r o p o r t i o n a l to the cross section o f the cell times { 1 - cos [(27r/~) aCD]). If the cell v o l u m e is k n o w n , e.g., b y a m e a s u r e m e n t o f the cell diameter, then the total scattered energy flux determines the protein c o n c e n t r a t i o n C. If the structural c o n t e n t o f different cells is comparable, as in the above cases, the equivalent w i d t h is a measure o f the diameter for spherical cell models. This can be seen f r o m fig. 4, where the cell diameter has b e e n plotted versus the equivalent width. In this case the equivalent width is p r o p o r t i o n a l to the cell diameter.

The a u t h o r in preparation paper is based for F o r s c h u n g

is i n d e b t e d to W.J. Rhodes for his help o f this paper. The w o r k on which this was s u p p o r t e d b y the Bundesminister und Technologie.

References [ 1 ] M.G. Kendall and A. Stuart, The Advanced Theory of Statistics, Vol. 1, (Charles Griffin & Comp. Ltd., 1969). [2] E. Evans, Opt. Commun. 2 (1970) 317. [3] R. Bracewell, The Fourier Transform and Its Applications (McGraw-Hill, 1965).