Automatic recognition of human face profiles

Automatic recognition of human face profiles

COMPUTEI¢ C~RAPIIICS AND IMACIE PIIOCESSIN(~ 6, 1 3 5 - 1 5 6 (1977) Automatic Recognition of Human Face Profiles L~ON D. H ~ m m ~ Department of Bi...

1MB Sizes 0 Downloads 0 Views

COMPUTEI¢ C~RAPIIICS AND IMACIE PIIOCESSIN(~ 6, 1 3 5 - 1 5 6

(1977)

Automatic Recognition of Human Face Profiles L~ON D. H ~ m m ~ Department of Biomedical Engineering, Case Western Reserve University, Cleueland, Ohio 4/f106 AND WILb~]~D F. HUNT Ohio Nuclear~ Incorporated, Solon, Ohio ~]~139 The ~tlmof (,hispreliminary profile-recognil.im~s ~udy was to develop algorithms for defining m~ique individual descripl.ion vec,~ors from mmmally entered profile t;races. The profile phol~ographs of ~t popular;tim of 25(i m~,les were reduced t,o outline curves by a~ art;ist and then scanned into a mMcomputer, A net ¢ff ~mt,om~,t,ieally derived fidueiaI marks, lines, ~mgles, areas, and ~ther mea,sures of the profile tr~ces was developed ~,nd refi~ed. St~i,istic~l mc~tsnres of the resultant, vector (~agsh~r t,he population showed l~hnt unique individmfl di fferent,i~Ltionwas clearly obl~aincdfor e~mhlnember of the population. ])istM1)ul;immand confidence measures were calculated for all feat,ures. The developmeut, of e()mpletely atttomai,ie sys~elns to identify, classify, sUore~ m~d reixieve images of hmmm fiw,es for modcsl, pol)uhtlJoa sizes m)w ~ppet~rs to be fe~sibie. 1. INTRODUCTION H u m a n identifie,~tion of hum-m faces is remarkably accurate, rapid, and inexpensive. Can we get a machine to compete? During the last few years there has been a growing interest in perceptual aspects of h u m a n faces, e.g., [1, 3, 4, 6-8, 10, 15-17] ; and a number of a u t o m a t e d or semi-automated recognition studies have been undertaken, e.g., [2, 5-7, 11, 12, 14]. Automatic identification, classification, storage, and retrieval of h u m a n faces could have considerable utility in m a n y personnel, commercial, security, and law-enforcement applications. Machine manipulation of information t h a t sufficiently represents a pictorial scene as complex as a h u m a n face is generally beyond ~he present s~gte-of-the-~rt. T h e usual approach is reduction of continuous-tone pictures to outline representgtions, e.g., [12, 14], or design of imergctive systems where humans machine-enter descriptors, e.g., [-5-7]. Another way to approach automatic identification of h u m a n faces is by way of profile silhouettes. This has the advantage of dealing with a relatively simple d a t a representation, a rather small, two-dimensional, binary-valued matrix. T h e sole example to date of such an approach uses circular autocorrelation techniques [11], an outgrowth of earlier developments based on m o m e n t analysis [2]. Two135 Copyright ~ 1977 by Academic Press, ine. All rights of reproduction in any form ra~ervod.

1.36

tIA]],MON AND IIIINT

Fro. 1. Two similar wavefl~rms.Judgments c,fsligh~differem~eswhen examined in this erientatioa are slow and tedious, and verbtfl dest.ripl.i,m is difficult. Belch ]udgmenls and ease cff reporlJng improve when traces are viewed wil;h 9(P ch,¢;kwlsembt~l.im~and tu'e perceived as htces. dimensional binary profile images of the frmlt perti(ms of times were used ; ~(laptive training decision rules were developed for ¢']f(~llt;1]]'(}'' {~()]~I])(}]I(}II'I;S ()[~ ch'cuhu' ~mtecorrelation functions. The profile recognition research reporeed here in based on quite different strategies. We consider how to represent, tmalyze, and elassil'y disc'rel, e features of outlines of face profiles. The problem then beeomc;s one of "wave, form" recognition similar, for example, to some of tim tmalytiet~] apl~roachcs for eleetroeardiogram gratings [9, 13-]. I~ is always tempt~ing ~o wonder, "IIow does the human do it?" [Nually l;his is at least unproductive, and at most it may lead to suspected brain-computer similarities which are unprovable. Despite this, it, is tempting to note a few introspective observations. Consider, for example, Fig. 1. Suppose you are asked to say whether these tracings are the same or different, and if different, how? The search and verbal reporting generally stresses comparable small segments and notes distinctions in slopes, peak excursions, etc. The process is rather plodding and tedious. Turn Fig. 1 clockwise through 90 ° so that the right side becomes the bottom. Now you perceive two face profiles, obviously and immediately different. Furthermore, it becomes far less tedious to say how they differ; there are clear distinctions in forehead bulge, nose shape, and chin protrusion. Perhaps the organization of subsections of profile traces into similar "features" will be useful for automation; at least, this is where we begin, eschewing for the moment holistie approaches such as correlation and template matching. However, while humans easily differentiate small shape differences in features (like lips or chins), automated techniques to make similar judgments are not necessarily easy. The detections and decisions described below are ad hoc evolutions of machine algorithms; they are suggested by human performance but are tailored to and constrained by rather uncomplicated computer techniques. It is well known tihal:familiar ob]eeesin unfmniliar posit~ionsare of(;elttmalyzed by conscious appraisal of bits and pieces, whereas viewed in normal a.spee~s, such obieots are perceived mm'e holist.ieally--at leas~in larger chunks such as nameable features (el. Fig, 16]).

137

RECOGNITION OF FACE PROFILES

) liqc. 2. Three progressively less informati~re representations of *t hum~m face. The Lask se~ by experiments repori;ed here is, "How useful is the line representation for individu,~l identification?" 2.

:~':EATURE SELECTION

Consider a f~me profile, f r o m forehead to throat, derived from a silhouette representagion of ~ conventional optical image, as illustrated in Fig. 2. There are several readily defined unique points which will serve as fiduci~l m a r k s for fe~tture construction. Obvious ones are nose and chin tip, nose bridge, etc. Eighg such independent points are sehemr~tized in Fig. 3. T h e u p p e r m o s t point is not independent but is simply a reflection of the chin point distance f r o m the nose ~ip as a center. Its utillty will become clear in subsequent discussion. lVlost el' these points are rel~tively time inwu'iant for s m a t u r e f~me. I-Iowever, the chin point c,m v a r y with lower mandible position, and t h e three lip points s.nd the thrower inflection are obviously mobile. Consequently, those five fiducisl m a r k s will h~ve to be used and ewfluated with care.

..Na--

/-

J

FIG. 3. Nine fiduei~fi m~u'kswhich eml readily be plrmed atttom~t,ieally. Eight ~t'e indepe,ndent; I~hetopmost lnsu'k is obtrdned by swing ng the chin-point mark tu'otul.d the rinse-tip m'u'k to intersecl~ l,he (xace ~t) Lheforehe'~d.

138

HAR~ION A N D H U N T

3

Fro. 4. :Examples of distance measures between fiducial pairs which can be used as vectm' ~omponents ~o identify individuals mliquely.

The first obvious use of such loci is definition of distance me;~sures which (ideally) will be inwu'iant for a given face. Figure 4 shows those used in the present study. Other uses of these fiduei'fl points ~u'e the definition of angles, of derived distances, and of areas, as suggested in Fig. 5.

Fla. 5. Other veeLor cumponents for representing profile traces are m~gles~ derived distances, and areas.

I~ECOGNITION

OF FACE

PI~OFILNS

139

These kinds of measures provide the basis for generating a multidimensional vector which is expected to be unique for each face. Questions will then arise as to just how unique, how reproducible, and how relatively useful various vector components are. Additionally, as will be described below, we introduce other measures in combination, such as line "wiggliness" and some line-shape correlation. Simplifying assumptions made for the moment are that hair is ignored and that the soft-tissue indicia (e.g., points 6, 7-9) and mobile point 2 are in neutral positions (i.e., mandibles in contact, head level, lips closed and relaxed). Given this, what tan be shown? The following sections describe how the approaches outlined above can be applied to real faces. 3. EXPERINIENTAL PI:tOCEDURES Profile photographs of 256 males were reduced to outline curves by an artist. Tiffs population of faces was taken l'rom an earlier pair of studies [6, 7] which used "~ population deliber~tely made homogeneous (white males, aged 20-50, beardless, no glasses, no obvious abnormal features). Photographs of the artist's tracings were scanned by a high-resolution rotating drum scanner (Optronics P-1700). After analog-to-digital conversion the curves were stored on a magnetic tape for subsequent processing by a minicomputer (DEC P D P 11/45). Output plots were produced by a CMComp plotter. The digital representation of each face outline w.'~s on the average ~bout 300 samples high, the sample having been derived fl'om the outside edge (right) o[ the artist's tracing. An example of 10 faces from the population, which provided one data sheet for scanning, is shown in Fig. 6.

Feature Extraction A, Fiducial Marks In order to obtain a set of defining "features" for each profile, nine automatically determined fiducial marks were used. 1. 2. 3. 4. 5. 6. 7. 8. 9.

Nose tip. 7 Chin. ]~ Basic Tri~ngle Forehead. Bridge. Nose bottom. Throat. Upper lip. Mouth. Lower lip.

These nine fidueial marks (see Fig. 4) were obtained as follows:

Nose tip and chin (I and 2). The nose and chin pair are the first fiducials located. A midprofile reference point is found by selecting the right-most point on t,he profile. Then with that reference, the line tangent at two points to the exterior of the lower half of the profile is found. These tangeney points are recorded as the nose and chin fiducials.

I40 HAR~'[0N AND HUNT

(

..J

Fto. 6. Ex&mple of 10 faces from a populatiou of 256, These tLrtist's l~raclugs of phobw'aphs were scanned into computer syst;em.

Forehead (8). The 1rose-chin distance is e~lcul~ted (R12), The point above the nose with a distance from the nose closest to R12 is recorded, ms the forehead fidueial. Br#lge (~,). A line is "drawn" from the nose tip to the forehead. Then the point between these fidueials which is most distant from the nose-forehead line is recorded as the bridge of the nose, Nose bottom (5). The line intersecting the nose-tip fiducial ~nd tangent to the interior of the profile just below the nose is found. The poin~ of tsngeney is recorded as the nose-bottom fiducial. Throat (6). The line intersecting tile chin fidueial ',rod tangent to the inside of the prone below the chin is found. The tangency point is recorded as the throat fiducial. Lips and mouth (7-9). Starting downwards from the nose bottom, a~ record is made of the radius of curvature of the profile for each point, successively. This continues until a pattern of three minima of the radius of curvature has been found with the curw~ture alterm~geIy to ~he inside, ou~slde~ inside, These extreme points are fl~en designated upper lip, mouth, and lower llp.

I{ECOGNITION

OF FACE

PROFILES

141

B. Characle'H~'tics Defined by Fiducial Marks The nine fiducial marks were used to generate six ldnds of feature characteristics. 1. 2. 3. 4. 5. 6.

Protrusion of nose. Area right of base line. Base angle of profle triangle. Wiggle (Hi and Lo). Distances between fiducials. Angles between fiducials.

These six characteristic feature elements (see Fig. 5) were obtained as follows:

1. Protrusion of the nose (P). A line is drawn from the chin to the forehead. The dist~mce from the nose fiducial point to this "baseline" is P. 2. Area right of baseline (AR). The aret~ defined by the Chin-forehead baseline and the profile curve is recorded as AR. A ~ is an algebraic sum of areas where the profile to the right of the baseline is (q-) and to the left of the baseline is (--). 3. Base angle of the profile triangle (ANG). The nose, chin, and forehead fiducials determine the Profile Triangle. Bec~mse of the way the forehead fiducial is defined, this is an isosceles triangle. Tim base angle of that triangle is recorded as ANG. 4. Wiggle. At erich point oa the profile one can determine a radius of curvature. This measures how abruptly the curve changes direction at that point. The wiggle parameter is the average of the inverse radii of curvature for points over a given region of the profile. This parameter is intended to be a measure of the vari.~bility (bumpiness or kinkiness) of the profile within a specific region. Wiggle-HI is measured between the forehead ~md the nose. Wiggle-LO is between the nose and the chin. 5. Distances between fidzedals. The distances from the nose tip to the bridge, nose bottom, mouth, chin, "rod throat are recorded as characteristic parameters of the profile. Also recorded is the distance between the upper and lower lip fiducial m~rks. We shall call the (Euclidean) distance from fiducial mark m to fiducial marl< n, Rmn. 6. Angles between fiducial~. The polar angles of all fiducials in a frame of reference with origin at the nose tip and baseline extending out through the chin m a y also be determined; however, they were not used as characteristics in the preliminary study. Feature Vectors In order to rapidly check an "unknown" individual against the entire population, a set of features was selected and a vector was generated for each individual of the population. After preliminary informal evaluation of a number of feature characteristics such as those described above, a list of 11 was adopted. These feature-vector components were: (1) Protrusion, (2) ~rea, (3) angle, (4) wiggle-HI, (5) wiggle-GO, (6) R12 (nose-tip/chin-point distance; sea Fig. 4), (7) R14-, (8) R15, (9) R16, (10) R18, and_ (11) R79.

142

HARMON AND HUNT

Pairwise dissimilarity between individuals was measured by determining the length of the vector difference between the individuals in a normalized Euclidean space. The normalizing consisted of division of ea('h component by the population variance for the component, ~v (x,. -- y,)~ D~(X, r) = E i~l

Ora2

After checking the "unknown" against the population profiles, the population is rank ordered with the most similar profile at the top of the lisL

Curve Matching A direct measure of the dissimilarity of two profiles is the mean separation of the profile curves after they have been aligned aceording to a simph, procedure. The alignment is done by selecting two fidueial marks wbich are fotmd on each profile. The profiles are then translated until the first fidueials eoin(.ide, t tmn they are rotated until the lines from the first to the second fidueial in t)oth profiles coincide. A specific range on the profiles is then selected, and the mean separation is determined over that range. In practice the decisions were restricted to the nose-h)tvhead and nosc-ehin fiducial pairs. A pair was chosen for alignment, and a pair was chosen to define a matching region. Each of the four possible cases was tested on the entire population of 256 individuals. 4. RESULTS The first step after profile scan and computer entry is computation of the nine fiducial marks. A check on these data is readily ohtained by plotter output. Figure 7 shows typical results for 10 subjects. The three tick marks of Fig. 7A are those first derived : nose tip, chin point, and its reflection to the forehead. The nose bridge, base, and throat ticks of Fig. 7B complete the gross feature comput-ttions; and the three marks of Fig. 7C show the lip-mouth decisions. These were checked in all subjects for reasonable, informal human concurrence ; no essential disagreements were found. The enlarged plot of Fig. 8 shows all nine of the fiducial marks superimposed on the profile of subject #1210, the "Mr. Average" of our population (who is depicted in Fig. 2 and described more fully in [6]). While the coarseness of the spatial sampling is quite evident, the satisfactory placement of the tiek marks in all 256 subjects indicates a sufficiently fine representation. (ttowever, it. probal)iy is marginally adequate.)

Feature Vectors An instructive first use of the feature vectors was an exercise employing distance measurements in scaled-Euclidean space. The system was asked to identify the most similar pair of profiles, the least similar pair, and "Mr. M e a n " - - t h e profile whose vector was closest to the vector of population means.

RECOGNITION

OF FACE

143

PROFILES

A

I B

~D

C

)1 (-

c~ Q

f

o~

FIG. 7. Typical outpu~ plot for 10 subiects. Used to check accuracy of computer algorithms in placing fiducia! ti¢.k real'ks : (a) nose tip, chi~ point, and reflected chin point, (b) nose bridge, bas% and throaL it~flcction, (c) lips Lind mouth.

144

HARMON AND HUNT

Fro. 8. All nine fiducilfl ticks plotted for subjcc~ # ] 210 who also appc~u's in Figs. 2 and 6. Note sp~tial quantizing effects on line-trace repz'esen~t~tioll (approximl~ely 300 samples wn'ticzflly). Figure 0 displays part of an ordered listing of most-simil'u' p~irs. Subjects 101.0 ~md 2808 (left-most pair) were said to be the mos g simih~r in the population of 32,640 p~irs. Subjects 1003 and 1005 were in second ph~ce on the ordered list. The result of a search for the most dissimila' p,~irs placed subjects 1303 ~md 608 in first place and Subjects 1.810 ~md 2002 in second ph~ce (Fig..10). Although c'asm~l inspection of Figs. 9 and 10 elicits reason~fl)le concurrence, t~here is no compelling reason for hum~m ;rod compui;er ,iudgments to t~gt'ee. Wtmt

o I-a O

rD rn f:~ CB

c~

¢2

0~

Fro. 9. Left: most similar pLtir in popuh~tkm of 256 profiles. Right: second most, simihu' pr~ir. Determinations were made oil basis of scaled Euclidcall disttmces in 11.-dim~l~sionalfe~zturc-vcctor apace.

145

]IECO(INITION OF FACE PIIOFILES

/ co

ffl

o

O~

F0

r~

c~

Fro. 10. Lerl~:le~ml,similar pMr in populat,im~(ff 25{1profiles, Right: second le~u~tsimilar pair. the computer decides as different (or similar) is not necessarily what the human decides since large components of our human iudgments are based on feature shapes such as nose and chin contours and forehead slopes, while such shapes arc not presently used in the machille analysis. Thus similarities in hum'.m and machine judgments must be taken here as fortuitous, though some aspects of tim computer's feature vectors (such ~s angle, protrusion, and some distances) indirectly relate to shapes as we perceive them. This observation bears on a perennial distinction (and sometimes problem) in man-machine decision-making considerations. Humans are very good at differentiating subtle differences in patterns, especially when shape Gestalts are involved; machines are poor at this. Computers, in contrast, are better at determining and using quantitative measures which are not necessarily "patterns" for human consumption. And so we usually take advantage of what the machine can best do. Although this may not necessarily yield the same sort of evaluation a human makes, the process may well be as useful (and accurate) in its own right. Consequently "similar" and "dissimilar" may often need to be understood as rather differently defined measures for man and machine. "Mr. M e a n " and his three closest competitors are displayed in Fig. 11. Subject 1106 was found to have a feature vector that was closest to the (hypothetical) subject whose feature vector components were the population mean wflues.

Independence of Feature Vector Components It is clear t h a t not all of the 11 feature-vector components can be expected to be independent. For example, protrusion and angle will be related for simple geometric reasons, as will protrusion and area. And measures like R18 probably are in some sense dependent on B12 simply because of the generic structure of the time.

146

HAIIMON ANI) I SUNT

/

C o o

01

n)

01 rn 01

03

~

Fro, 11, Left-l;o-righ(~: "Mr, Mean" and tJ~ree eloses[~ eom5)egil,ors il~ clec~l'eIMulgelos(~m~ss of individual's feat,ure vector 1~ot,he me~m po5)uh~i.i
T~ble I shows the 11 X 11 correlt~tlon m a t r i x for the I'(,,aI~ur(:-vecl;or components, and Table I I provides a rank-ordered listing of all p~irwi,~c :f(~tur(~ c o l relations whose m a g n i t u d e is grea~cr them 0.2, F o r t h e s a m p l e size use(l, this r o u g h l y will include ~11 pairs significan£1y correlated ttt t h e 0.05 cot~:fiden(~(; level. T w e n t y - t w o of the forty-five pairs are so included. As antici[mted, t h e dep(;ndenl; relationships among protrusion, "~re~b and 'ingle r~mk high, a n d the. R I 2 / R I , S cot'r e l ~ i o n of 0.59 is not surprising I t seems likely t h a t several feature-vector c o m p o n e n t s c o u l d be oliminated w i t h o u t greatly diminishing individual identification. A systc, mt~tic test, ing of feature elimination to maximize perform~mce for m i n i m u m c(.)mputttt, i(m effort, should be one of the next investigative steps. TABLE 1 Correlation Coefficients of 11 Profile Features ANG1 PROT AREA WIGU WIGD ~ 1 2 R 1 4 ANG1 PROT AREA WIGU WIGD 1~ 12 1~ 14 1"t15 ~16 1~18 ~79

1.00 0.86 0.74 0,07 0.03 0,16 0.23 0.24 0.05 0.28 0.02

0.86 1,00 0,92 0.11 0.10 0,64 0.27 0,34 0,28 0,52 0.08

0,74 0.92 1.00 0.09 0,06 0.67 0.18 0.16 0.27 0,43 0,11

0,07 0.03 0.11 0.10 0.09 0.06 1.00 -0.25 -0.25 1.00 0.09 0.16 0.03 0.03 0.07 0.03 --0.01 0,15 0,01 0.14 --0.02 0.02

0,16 0.64 0.67 0.09 0.16 1.00 0,18 0.30 0.50 0,59 0.13

I~15

0,23 0.24 0,27 0,34 0.18 0.16 0.03 0.07 0.03 0.03 0.1.8 0,30 1.00 0.22 0.22 1.00 0.12 0,17 0,17 0,30 0.05. --0.00

2t. 16

R18

R79

0.05 0.28 0,27 --0.01 0,15 0,50 0.12 0.17 ~1,00 0,32 0.10

0.28 0,02 0,52 0.08 0,43 0,11 0.0[ -0,02 0,14 0,02 0,59 0,53 0.17 0.01 0,3(i --0,0(i 0,32 0,10 1,00 0,51 0.5I, 1,00

RIiXX)({NITI()N OF FACE PROFILES

147

TABLE 2 (~m'relat,i(mRmfidng, Cul,off = 0.2 Feal,ure pMrs

Correh~t,ion

PICOT ANGI AN G t AREA PrCOT 1~ 12 PROT R 18 It 12 A~R,'I,IA

AREA PROT AREA R 12 l:t 12 R 18 R 18 R 79 R l(i R 18

0.92 0.86 0.74 0.67 0.04 0.59 0.52 0.St 0.50 0.43

R 15

R 18

0,36

P].~,OT It, 16 1t, :12 PIt,(IT AN(It AI{],,'A Pit( )T WI(itr AN( I1 AN( I 1 1~ 14,

R ] ,5 R 18 R 15 I1, I(I ]L 18 R 1(} R 14 WI(ID 1~ 15 1~ :14: R 15

0,34 0.32 0,30 0.28 0,28 0,27 0,27 -0.25 0,24 0.23 0.22

Di,stribulio.n,s, of Fealure Values Normalized histx~grams showing the distributions of the 11. features over the population of 256 subjects appear in Fig. 12. Each range was normalized to t~ ± 3¢. Most of the features appetu: to be 'q)proximately Gaussianly distributed. However, three seem s o m e w h a t exeeptionaI, The R79 or lip-spacing measure is strongly peaked, the relatively small deviations suggesting that in future study this feature m a y not be very valuable. T h e two wiggle distributions tend to suggest the same, but if W I G - H I and W I G - L O are not highly correlated in the same individual, ~hen the measure can still be useful. This will be discussed further in a later section, N o t e t h a t both W I G s are skewed to the high end; no profiles ~re circular, most are m o d e r a t e l y wiggly, and quite a few tend to be rather convoluted. To avoid the information Ioss inherent in histogram bins and to experiment with some new display ferments, we generated the plot of Fig. 13, Each feature value over the population was entered separately ; again the scales were normalized ~o ~ 4- 3¢. T h e result is a m o r e easily grasped picture of distributions.

Interindividual Distances Finding tile feature-vector separations among all members of the population is critically i m p o r t a n t to individual identification. Ideally, each individual's Euclidean distance from every other individual should be large enough to permit

148

HARMON AND HUNT ¢D

J

F--i

O -e

-"["1

I

I

I

I

I

I

I

r

'J

I

I

I

[

J

I

l



I

r

i'

I

I I

L

l

I

r h~ ~Z

t~

I

I

I

I

I

I

'f'

I

I

I

'1

I

i

J

I

I

I

I

~

i

I

I

E

i

I I i

i

[:i

I

~ d

.,

v~ ~E

J

I

I

I

a~

I

,

~9

Z

C

1

i

.1

II ,,I I ....

[- . f l

[ J i l l

O

!

'

II Il

t

[ F(:3 fi: ~L

~3 fz

I

I

I "1

J

I

I

f ~f

'

o

o

eb

RECOGNITION

RIB

UI

If

R?9

/%

11

I I ttl

I

Rl~

Z~

R12

~

z~

/~

I I:

Ill

Z~'

A

ANG1

~r tl

IIH/I~I

Z~

A

Ir

T

,,

r

A

A

A

lllllllililllll~glHflill

2 IIIIIIIIIIJIIJlL

r

....

I,II~_~ll]~l,llt~i|

IIII1!1 lip It ~! I

I I

IIIHIHtlll t,t iIIIIIAII

i II

A

IJLllllill lltllllltlll

Z~II~IImIII~LI~LZI]U.IL~LLLW.IILU_UAI

wIGu

PROT

II IIIll~al

t!~llill

~IGD

AREm

IIHtl

I ~111 frill tlIILJ~LILLLLtlll

it I [ f llll Ill I IItl I1 lit II [JLILIIIL]I]LLLI~I_LJLLLL~~I'L_]J]LJI..LIILI,. III Ill Ill[ al I[ r

I

I

149

III£LII IIIIIIIIIlIIILL[~IJI~I~I~IIlIIIIIIIIIII[liII~IIIIItlIII]I

z %

R~. 5

OF F A C E PROFILES

,, m - . l ~ i ~ l l l L ~

'

'"'"'"'

'

I

A

,~,

! II1!1 Ill

I

Z~

lll l

/A

ml I

[E

I

Zk

Z~

'

"

z~

'A

Z%

1~..~iI

Z~

I Ilia! I Iflli II:II::D;'IIIIHFHII~ ~' IfF~llll':fl!!llfll!ll~llllll A A A

I

P!I II Ill ZX

~.

f

Fro. 13. Actual sample-value distributions of features. Each is normalized to ~ ~ 3~. Spacing ticks are at one-~ intervals. Compare to Fig. 12.

robust identification , even in the presence of noise (such as erroneous measurements, lip a n d jaw displacement, etc.). And even if clean separation between the target s u b j e c t and the rest of the population cannot be obtained, the t a r g e t should ,~ppear high in -my rank-ordered list of "possibles." As in m a n y pa~ternrecognition cases, reliable pm'titioning of the alternative identifications into a

].50

I-IAII,MON AND tIUNT

rdatively small subset of "best bets" is ofl,cn useful wh(m al)sohtl;e ident, ifi(;t~tion canno~ be obtained. :Figure 14 summarizes the target-separ~tion results for the experiments reported here. All pairwise distances were computed for the population. Thus, 256~/2 -- 32,768 were found. The normalized his¢ogr~ms in the figure are meant ¢o convey ~ qualitative impression ef sep~ration. Dettfiled qtmntit~Ltive analysis is reserved for future study where severtfl s~mp]es of erich individtufl t:tken ~Lt different times are avaih~ble so t h a t the tests become more realis(~ic. The top plot of Fig. 14 shows the cross-vector histograms for the entire population. Self-m~tches t~re seen as the single spike ~g the origin. The clever space between that spike and the cross-match distrib~r~ion shows tlu~t ,'~ simple threshold

o/ I

I

"1'

1

I

l

|

I

1

I~1

I

I

I

I'

I

I

I

["

I

1

J""7'

I

,

I

I

I

1

1

l

I

FIe. 14. Cross-vect:orhistograms (11 features). Top: All interindividu,tl distances. For each individual, the Euclidean distances t.o ~filindividu~fisi,1 the popul~t,ion t~re compul;cd. The plol. of l~he resultant 32,768 enh.ics include the 256 self-matches ~ t,he origim Center: l~rgcs(, dist~mces. :For each individual, the mos~disi~antpopuh~tionmember is found, ~Hd I~heresulI,[ml,258 disinclines are ploi~ted. Boi,l;om: sm~llesbdisi;anccs.

]IECOGNITION OF FACE PROFILES

151

decision for this e x a m p l e would suffice for unequivocal identification of all 256 individuals. T h e center plot displays the best cases of separation for all 256 p o p u l a t i o n m e m b e r s , and the b o t t o m plot shows the worst cases. Curve Matching

W h e r e a s featm'e vectors generally provide a flexible and open-ended identification procedure, c u r v e m a t c h i n g (literally t e m p l a t e matching) is m u c h m o r e restMcted and u s u a l l y less potent. Yet the results obtained in this s t u d y with c u r v e fitting t e n d to be encouraging.

~ll|"'[[ll]Itlll[(llll FIG. 15. Populatien cross-fit residues. For each pair of profiles in the population, the mean difference is determined for the nose-to-chin portions o[ the curves. First, the two profile curves are brought into stm~dard correspondence by a shift which makes nose tips coincide, then a rotatiou until the nose-tip/chin-point lines coincide. The mean difference or "residue" is then computed between each pair of nose-chin curve segments. Top : All pairs. Center ; largest residues. For each individual, the largest residue in the population is found, and the resultant 256 values are plotted. Bo~{;om: smallest residues.

IrAIIMON AND HUNT

152

:Recall that p~irs of profile curves were matched, using nose-:forchc,,ad (R13) and nose-chin (R12) fidueial pairs. Consequently four eases were tested: (1) ma%h R 1 2 t~duciMs, find 1712 curve diffe, renees (Pig. 15), (2) mat;c;h 1~,12 fid~dals~ find/~13 curve differences (~'ig. 16), (3) match R13 fidueials, find R13 curve differences (Fig. 17), and (4) match I~13 fiduc, i.~.Is~ find R12 curve differences

(]~ig, 18). In each of these four figures, three results are displ'~yed, the top showing the cross-fit residues for nil population pMrs. In t,he center can t)e seen t,he distributiort for the largest residues (one for each member of the popult~tion), while the boO,era plot displays the smallest residues, i.e., the worsg cases where small residue means similarity or eonfusability. As might; be anticipated, Figs. 1.5 and 1.7 arc; similar, as are Figs. 1(~ and 18. Thus if nose-tip/h3rehead fiduc'ials t~re lined ttp and elu've, ctil'l'erenc~;s arc; l;akea

T I 1

I 7" T T T ~--1--1-"1" 1" T 1

,kn

~" z

I

i

I

I

I

I

~1

I~"1-~I

T--1

I l l l l l

I

P"I"

I

I--I

FI(~. 16. Pop lh~tion cross-l~l, residues for nose-(,ii)/forehcacI pm'l.icms ~,f cmrvc,,swhen l,he, n¢~sctip/c~hin-poin~ fil;s are made ~s in Fig. 15. Top: AIL p~tirs. Cenl,er', hu'ge.~l~ rcsichms, Ball,ore'. slnMlesl; resldues.

1-1,I~COGNITION OF FACE PROFILES

l

I

l'-i

] ]

I-I

I

I

I

I

l

I

I

r

I

i

I

I

i

I

I

I

I

I

I

I

I" I

I

153

I"[

I 'I""I'~1"I'~I

FI(~. 17. Populal;ion cross-fit residues for nose-~ip/foreheadportions of curves when the nose~ip/forehead-point fits arc mMe. Top : all pairs. Center : largest residues. Bottom : smallestresidues. for that upper half of the face, the results are roughly similar to those obtained for the same operations on the lower half of the face. And if the lower half is aligned while upper-half curve differences are taken, the results resemble those where the upper hMf is aligned while taking Iower-half differences. Let us refer to the first ease (Figs. 15 and 17) as "same-half" match and to the second case (Figs. 16 and 18) as "other-half" match. The same-half matches yield tight population cross-fit residue distributions while the other-half matches produce much broader and highly skewed distributions. Similarly, same-half matches show best eases (largest residue) which are both more tightly distributed and smaller mean valued than are the other half matches. Worst cases (smallest residues) are reasonaly similar in all four cases, being sharply peaked and close to the origin, imp]ying confusability. All of the foregoing suggests that some utility can be expected for other-half residue evaluation. This could be a useful adjunct to the feature-vector approach,

154

HAR~'ION AN]3 HUNT

j

I

r

i

i'lrYli

i

ln!ll

1

9

I

I

I

I

I

I

I

t

w

I

I'

z

t

Fro, 18. Population cross-fit residues for ~he nose-tip/ohin-poia~ portions of curves when ~he nose-tip/forehead-poing fits are made. Top : M1 p~irs. Center : largest residues. Bottom : smallest residues.

Moreover, it is no~ yet known how useful the residue results will become when individual differences are separately accounted rather than population lumped as in the preceding discussion. That is, individual A could have a close upper other-half fit to that of individual ]3, but their lower other-hal~ fits might be rather distinct, thus yielding usefnl discrimination. This has not been rigorously tested for the present experiment, although informal inspection of the data suggests that the approach might work fairly well.

Running Times The present time requirements to process profile traces were not optimized for economy. Emphasis was placed on algorithm development r~ther them on speed. Even ~o, the file structures and data manipulation employed probably requirQ 1~o

II,ECOGNITION OF FACE PROFILES

155

more than twice t~s long as a minimum-time system would require for the same hardware configuraSion. Ten pictures at a time were optically scanned, taking roughly 10 rain terM. The resultant tape was processed to a thinned, final-samples tape representation in 20 rain. (This was done by an all-purpose general edge-detection/boundarylinking routine which is useful for processing X ray, biological cell, and other images to locate boundaries.) Each picture thus required 3 rain for initial permanent-file input, to the point, where actual processing for identification begins. Next, fidueial-mark computation required 3 sec per portrait. Then feature extraction (angle, wiggle, protrusion, etc.) required another 3 see. The vector file thus gener~ted took a total of about 3.1 rain, the bulk of the time being due to scanning and spatial-sample conditioning. Much of this time, consumed by spinning tapes, could be saved by using core or disc. The cross-vector comparisons for the population (256 X 255)/2 = 32,640 measures required about 1 hr. Obviously, file search procedures such as this can be refined, particularly if large populations are initially separated into subclasses. Given the present time, however, '~ lJle match for one "suspect" in a population of 256 would take 3600/256 "~ 14 sec of computing. The cross-fitting program, wherein residues were computed for four kinds of fit, took 7 hr to execute for the population. Two fits could suffice, taking 3½ hr, and undoubtedly more economical data handling could be devised. Still, this is a time-consuming operation. Given the present rate, a suspect could be cross-fitted to all members of the population in 420/256 ~ 1.6 rain. This is rather long compared to the 14 sec (or h,ss) n e('d(,d for vector comparison. 5. CONCLUSION Tile feature-vector components used in this study were hi)solute magnitudes (distances, areas, :ingles). Since tile portraits initially were made at a standard distance, such representai;ions of distance and area can be used with confdence. However, if there is no such control, and relative measures are all that can be expected, then ratios rather than magnitudes should be employed. Similarly, normalizing can be employed in the curve-matching procedures. The present technique superimposed nose-tip fidueials, then rotated curves until nose-chin or nose-forehead lines coincided. A more effective procedure might be to map one curve into the other by scaling such that both fiducials coincide; residues would then be taken. Some of the measures used are obviously more robust than are others. For example, nose-tip and nose-bridge fiducials and wiggliness are relatively invariant compared to measures such as chin point or lips which can vary with jaw position or soft-tissue motion. Cooperative subjects can be counted upon to present a relaxed, jaws closed, neutral and standardized posture; uncooperative or unalerted subjects can cause considerable variation in some of the measures used at present. Study of the relative utility of the feature vector components is indicated; a probable outcome is differential weighting of some of the variables, elimination of others, and acquisition of some new measures.

156

HARMON AND HUNT

H~dr presents t~ speci~fl problem, pm'tieulm'ly with t o d ~ y ' s f r e q u e n t hirsute adornments. Possibilities for coping w i t h beards a n d m o u s t ~ c h e s include replacing optical i m a g i n g b y u l t r a s o u n d or e v e n X ray. Skull pro'traits w o u k l require a different set of measures, b u t t h e y should be quite effective. REFERENCES 1. R. A. DarL, The wal~erworn Aust;ralopitheciue pebble of many faces from Makaptmsga[;, S. Aft. J. Sci. 70, 1974, 167-169. 2. S. A. Dudani and K. J. Breeding, Aircraft idenl,ifical,ion 1)y momeltt inwu'iauts, IE1,.,'E Trans. ffomp., C-26(1), 1977, 39-45. 3. H. D. Ellis, Recoguizing faces. Brit. J. Psych. 66 (4), 1975, 40.q-426. 4. J. F. Fagan, III, Infants' recognition of faces~ Chihl De~;elopment, ] 976, 47, 627-638. 5. M. L. Gillenson and B. Chandrasekaran, Heurisl,ie s~rategy for developing human facial images on a CRT. Pattern Recognition 7, 1975, 187-196. 6. A. J. Goldstein, L. D. Harmon, aud A. B. Lesk, Idenl>ification of human faces. Proc. IEEE 59(5), 1971, 748-760. 7. A. ft. Geldsteiu, L. I). Harmon, and A. B. Lesk, Mau-nlachine ilfl,cra(,(,flm in hunmu-face :ideutificatiol b Bell Sys. Teeh. J. 51 (2), ] 072, 3(,l.q-427. 8. L. D, Harmon, Some aspecL~ of recog~fil,ion of human faces, :il~Pattern Recognition in Biological and Technical Syslems (O.-J. Grfisser, Ed.), 1)P. 196-219, Spr|nger-Verlag, New York, 1971. 9. L. D. Harmon (Ed.), Biomedical Signal Analysis, roper(, (ff a w(n'kshol) held at Case Wesi,eru P~eserve University, Jammry 12~-14, ]975. The Nat,iomfl Scien(,'e Foumlal.ion, Washington, D.C., 1975. 10. 3. Hochberg and R. E. Galper, Reeogni(~ioll ()f faces: I. An explora(~m'y study, Psychon. Set. 9(12), 619-620, 1967. 11. G. 3. I(auflnan, Jr., and K. J. Breeding, The automal,ic recognii.i(m of lmman faces front profile silhouettes. IEEE Trans. Systems, Man, Cybernetics ~.gM(J-#, ] 976, ] 13-121. 12. Y. Kaya and K. Kobayashi, A b~usicstudy on humm~ face reeogtfil.ion. In: Frontier's of Pallern Recognition (S. Watanabe, Ed.) pp. 2(15-289j Academic Press, New 'York, 1(.)72. ]3. F. M. Nolle and K. W. Clark, ])etecl.ion of premature venl~ricular (;ol~tracl.ious usil~g an algorithm for cataloging QI%S complexes. Prec. San Diego Syrup. Biomed, Eng. 10, ]971, 85-97. 14. T. Sakai, M. Nagae, and T. Kauade, Comput.er analysis and classificai.itm of photographs of human faces. In: Proceedings of the Firsl USA-Japan Computer Gonference, pp. 55-62. AFIPS Press, Montvale, New Jemey, 1972. 15. 1%.:K. Yin, Looking at upside-down faces, J. Exp. Psychol., 81 (1), 141-145, 1969. 16. I~. :K. Yin~ Face Recognition: A Special Process. :Report P-4419, The New York City l¢and Institute, New York, 1970. 17. A, gavala, Pemonal Appearance Identification. Tech. Rept. PB 202032. CAL. XM-2814-B-2. Cornell Aeronautical Labs., Buffalo, New York, 1970.