Novel Level set based signatures for Gait Analysis

Subtitle as needed (paper subtitle)

Karthik K. V.*, Meghana Dinesh Kumar, Sunil Rao M. S., Channabasava Nucchha, Subhash Kulkarni.
ECE ES!"# $SC $angalore karthikvkal%ani&gmail.com*, meghanadineshk&gmail.com, sunil.msr'()&gmail.com, channabasavanuchha&gmail.com, sskul&*es.edu
Abstract Gait analysis has gained prominence towards detecting suspicious activity in public area which is generally kept under surveillance. In this paper we present the results of recognizing a person based on his gait movement. Novel signatures were built through distance mapped level set contour function based on boundary and skeletal information. Gait video sequence experimented upon is silhouette corresponding to 1 sub!ects with "# frames per sub!ect. $esults were found to be encouraging at first level where signatures have been compared with available sub!ects% gait video. &he robustness is tested with morphological distortions on the query image. &he sub!ect has been found to be detected with reasonable acceptance. Keywords Level sets, Boundary and skeleton signatures, Gait il!ouettes, "istance features#

Real time video has become reali<able in recent decade, due to advancements in various technological as*ects such as increase in internet band-idth and *rocessing ca*abilities. !n the recent decade, ,ait anal%sis has gained *rominence in situations like sus*icious activit% detection in *ublic *laces. "hough limited to -alking alone= it ma% be a**lied to other +orms o+ locomotion like running also. !n our *ro*osed method o+ ,ait anal%sis, -e use a de+ormable model based a**roach. Silhouettes are obtained +rom the real time motion o+ sub.ects obtained a+ter segmentation and background subtraction. >sing these silhouettes, signatures are derived as gait +eatures. 2ur research is +ocused on identi+%ing the sub.ect through corres*onding gait *atterns. !n their earlier -ork, 6ang, et.al : ; uses C0 1 rinci*le Com*onent 0nal%sis3 a**lied to time var%ing signals derived +rom silhouettes to reduce dimensionalit% o+ the in*ut. attern classi+ication techni7ues -ere a**lied +or Eigen vectors +or recognition. "he dra-back o+ this method is that it is viede*endant. 0nother method b% Sudee* Sarkar uses baseline algorithm com*rising '? e@*eriments and data set= the algorithm estimates silhouettes b% background subtraction and *er+orms recognition b% its tem*oral correlation : ;. Aimitation o+ this is the *oor silhouettes obtained +rom segmentation. Research re*orted b% Da-son uses C0 based a**roach +or dimensionalit% reduction. !t has a disadvantage that the camera lens direction is al-a%s *er*endicular to the direction o+ the *erson -alking. "here+ore, side vie- or *ro+ile vie- o+ an individual is re7uired. : ; 6e have e@*lored that there is a *ossibilit% to derive +ast signatures based on distance ma**ing +unctions built -ith boundar% in+ormation. Bast a**roaches to build level sets or distance ma**ing +unctional have alread% been *ublished b% the authors earlier. :Sandee* V. M;. !n the *resent -ork, -e have attem*ted gait recognition using boundar% signatures along -ith skeleton signatures that are +ound to be translation invariant and scale invariant.

I.

Introduction

"he *rocess o+ identi+%ing a living being b% manner o+ its locomotion is ,ait recognition and is more s*eci+ic to humans. !t is an unobtrusive method -here sub.ect/s gait under consideration can be ca*tured +rom a distance. !n disguise, the sub.ect ma% a**ear more sus*icious. "he ca*tured images need not be o+ high resolution as recognition is *redominantl% boundar% based. "his makes it advantageous com*ared to other methods, -hich are s*eci+icall% content driven methods. Scienti+ic gait anal%sis is attributed to *ioneering -ork o+ 0ristotle in De Motu 0nimalium 12n the ,ait o+ 0nimals3 and much later in '45), ,iovanni 0l+onso $orelli. 6ith the develo*ment o+ *hotogra*h% and cinematogra*h%, it became *ossible to ca*ture image se7uences that revealed details o+ human and animal locomotion that -ere not noticeable b% -atching the movement -ith the naked e%e. 0lthough much earl% research -as done using +ilm cameras, the -ides*read a**lication o+ gait anal%sis to humans -ith *athological conditions such as cerebral *als%, arkinson8s disease, and neuromuscular disorders, began in the earl% '(9)s -ith the availabilit% o+ video camera s%stems that could *roduce detailed studies o+ individual *atients -ithin realistic cost and time constraints. "he develo*ment o+ treatment regimes, o+ten involving ortho*edic surger%, based on gait anal%sis results, advanced signi+icantl% in the '(5)s. Man% leading ortho*edic hos*itals -orld-ide no- have gait labs that are routinel% used to design treatment *lans and +or +ollo-#u* monitoring. :-iki*edia;

II. Background
"he silhouette gait database -e have obtained is +rom C$!R, C!S!0, China. "he gait +rames re*resent movement o+ *erson +rom right to le+t as indicated in the +igure.

Bigure 'C Database

E@*eriment -as conducted on the database com*rising '4 individual video having D) +rames each. Sam*le silhouette +rame and corres*onding ED sur+ace *lot o+ distance ma**ed silhouette embedded level set +unction is sho-n in +ig.

Bigure EC 1a3 1b3 1a3C model curve F 1b3C distance ma**ed level set G 1c3C E D *lot o+ G and corres*onding HAS

1c3

III. Novel Boundary And Skeleton Signatures

Aevel sets are created on a +rame using distance ma**ed +unction and are *rimaril% collection o+ identicall% valued level contours. Aevel set +rame-ork is -idel% used in vision *rocessing in ob.ect e@traction based on geometric de+ormable model. Aevel set +unction is an intrinsic +unction embedding the model evolving curve as <ero valued level contours. 0ssuming F as the model curve, the corres*onding level set +unction G is a distance ma**ed +unction -ith res*ect to F. "he corres*onding HAS 1Hero level Set3 denoted b% IC/ can be obtained using the conditionC CJ K@L G J J )M 1'3

!n the *resent -ork, evolving the model curve is un-arranted, as the silhouette boundar% can directl% be re*resented as the model curve that can be intrinsicall% embedded into the level set +unction G. simultaneousl%, the boundar% also corres*onds to region o+ the ob.ect re*resenting segmented version o+ the image. $oundar% embedded level set +unction inherentl% corres*ond to several length based +eatures. "he +eatures that have been attem*ted in the *resent -ork are around boundar% length n) and skeleton o+ the silhouette. 0bsolute boundar% length is onl% suitable -ith ideal database, -here 7uer% is *art o+ the database. !n *ractice, variations are observed in the 7uer% image and database images in terms o+ translation, scaling, and rotation and %et more com*licated being ske-ing. !n the *resent -ork, to accommodate gait movement, deviating +rom the database, -e have used boundar% +eatures in a normali<ed -a%, -ith res*ect to boundar% length. "he corres*onding normali<ed +eatures re*resent the boundar% signatures as belo-C

6hereC

N#' J n#'Nn) NO' J nO'Nn) Nskel J nskelNn)

n)C number o+

*i@els on the boundar% n#'C number o+ *i@els -ith unit distance outside the region nO'C number o+ *i@els -ith unit distance inside the region nskel C number o+ *i@els l%ing on the skeletal image a+ter s*urring correction.

,ait +rame 0ctual skeleton S*urred skeleton Bigure DC Silhouette +rame and its corres*onding skeleton

S*urring corres*onds to erosion o*eration o+ a set 0 -ith a circular disk -hich is large enough to +it in that set so that it touches the boundar% o+ set 0 at t-o or more *laces. 6ork is going on to add more signatures that can increase scalabilit% and robustness

Skeleton is obtained +rom distance ma**ed level set +unction using the +ollo-ing conditionC Skeleton, S J K@ L @ J Gma@M nskel corres*onds to number o+ *oints on skeleton a+ter s*urring correction. Pere NO#' corres*onds to normali<ed boundar% signature o+ unit length outside and unit length inside the actual boundar%.

IV. Level Set Based Approach and Gait Analysis Architecture

"he *ro*osed architecture +ollo-s a t-o stage a**roachC "he +irst stage corres*onds to signature com*utation o+ ever% +rame in the database. 0n 5))@E signature +eatures are e@tracted +or the '4@D) silhouette +ramesC

!n stage ', +ollo-ing ste*s are *er+ormedC

Bigure QC Distance ma* method

'. ?. E. Q.

$oundar% is detected using Cann% edge detector. Distance ma* level set +unction is com*uted using boundar% in+ormation. Skeletal in+ormation is e@tracted. >sing boundar% and skeletal in+ormation +rom level set +unction signatures N#', NO', Nskel are com*uted.

Scaling to a certain e@tent is ensured to be invariant b% normali<ation. "o increase the matching e++icienc%, additional signature is used -ith normali<ed skeleton length o+ silhouette. "hese combinations o+ t-o $oundar% and one Skeletal signature are used in the *resent -ork +or gait recognition. "hese three signatures are com*uted +or all the 5)) images, %ielding a matri@ o+ 5))@E +eature matri@ stored in signature database.

Stage ? com*rises signature com*utation +or the 7uer% image +ollo-ed b% matching to %ield to* best matches. "he metric used +or com*arison is Euclidean metric, though other metrics are also available. "he ob.ective is to obtain signatures -hich are ma@imall% invariant to scaling, rotation, translation, etc. "his is obtained +or each o+ the +rames that are in the database and it is stored in the +orm o+ a matri@, corres*onding to +eature matri@. Stage ? involves similar *rocedure -here the 7uer% image and the all the three signatures are com*uted and is matched -ith the database obtained in *revious section. "he ob.ective is to obtain the signatures -hich are ma@imall% invariant to scaling, rotation, translation, etc.

V. Results and Discussions

E@*erimentation is conducted -ith similar ob.ective as in an% C$!R s%stem. !n the *resent -ork, the nearest match to the available 7uer% silhouette is tested. "esting is conducted -ith available 5)) +rame silhouette database. Robustness is tested b% sub.ecting the 7uer% silhouette to dilation, erosion, <ooming in and <ooming out. Results are +ound to be a**reciable. !m*lementation is done in M0"A0$ -ith 5))@E +eature matri@ built as described in the architecture, 7uer% image is sub.ected to +eature com*utations and to* 'D silhouette matches +rom database are dis*la%ed. "he *ro*osed algorithm is tested thoroughl% on a dataset o+ '4 *eo*le each having D) +rames. "he test made initiall% including $oundar% *i@els= ho-ever, the *er+ormance o+ the algorithm +ailed. !t is observed that initial boundar% signatures -ere n ), N#', NO' insu++icient in retrieving the *recise matched silhouettes. "o address this limitation, -e added another signature derived +rom the skeleton o+ the image. "his signature has increased the retrieval e++icienc%. "he advantages o+ three boundar% and skeleton signatures are as +ollo-sC "he undistorted 7uer% image 1an image *resent in the dataset3 gives the best match -hen com*ared -ith the dataset. "he 7uer% image -hen sub.ected to scaling 1including mor*hological o*erations3, *rovides a best match as the *ro*osed algorithm uses normali<ed signatures. "he resulting image is dis*la%ed -ithin 'R o+ 5)). Results +or all above situations are *resented +rom 1S3 "he $est Match result is the one -here the Euclidean Distance bet-een 7uer% and +eature matri@ is the least. Bor erosion, the results are +ound to be -ithin to* QR. R2$>S"NESS ,ait recognition is tested +or robustness b% sub.ecting 7uer% silhouette to translation and scaling, es*eciall% <ooming in and <ooming out, dilation and erosion. Bor dilation and erosion the image is sub.ect to distortion using a $all o*erator o+ m*n si<e -here m stands +or radius o+ the ball and n stands

+or the height. "he $all o*erator used is a ?@? dilator. 2ur e@*erimentation %ielded the correct match -ithin to* ') retrieved images. Po-ever, the *er+ormance deteriorated +or erosion, still retaining the correct match -ithin to* QR. "he result +urther deteriorated +or increased distortion +or more than ?@? mor*hological o*erator and <oom in 1T '.?D3 and <oom out 1T ).9D3. "hus our method has e@hibited invariance to translation and limited invariance to scaling +ig.

VI. Conclusion
>sing the database o+ silhouettes, -e have *resented three signatures -hich are invariant to translation and scaling. "hese signatures are derived using boundar% and skeletal +eatures. 6hen a 7uer% image is encountered, its three signatures are com*uted and the best match is +ound b% calculating its similarit% measure -ith each image in the database. Results are +ound to be satis+actor% and the *reliminar% +indings are encouraging. "he search is also com*utationall% e++icient. "he *resent -ork is no- e@tended to e@*lore additional signatures -hich can render rotation invariance. 6e are also -orking to identi+% uni7ue signatures that e@hibit a *attern similarit% bet-een di++erent gait videos. !t is observed to a limited e@tent, N#', e@hibits similarit% bet-een sub.ects -hich re*resent Unormal gaitV. 0n% sus*icious gait video is +ound to deviate +rom this *attern

Acknowledg$ent %eferences
"he tem*late -ill number citations consecutivel% -ithin brackets :';. "he sentence *unctuation +ollo-s the bracket :?;. Re+er sim*l% to the re+erence number, as in :E;Wdo not use URe+. :E;V or Ure+erence :E;V e@ce*t at the beginning o+ a sentenceC URe+erence :E; -as the +irst ...V Number +ootnotes se*aratel% in su*erscri*ts. lace the actual +ootnote at the bottom o+ the column in -hich it -as

cited. Do not *ut +ootnotes in the re+erence list. >se letters +or table +ootnotes. >nless there are si@ authors or more give all authors/ names= do not use Uet al.V. a*ers that have not been *ublished, even i+ the% have been submitted +or *ublication, should be cited as Uun*ublishedV :Q;. a*ers that have been acce*ted +or *ublication should be cited as Uin *ressV :D;. Ca*itali<e onl% the +irst -ord in a *a*er title, e@ce*t +or *ro*er nouns and element s%mbols. Bor *a*ers *ublished in translation .ournals, *lease give the English citation +irst, +ollo-ed b% the original +oreign# language citation :4;.
:'; ,. Eason, $. Noble, and !.N. Sneddon, U2n certain integrals o+ :?; :E;

:Q; :D; :4;

:9;

Ai*schit<#Pankel t%*e involving *roducts o+ $essel +unctions,V hil. "rans. Ro%. Soc. Aondon, vol. 0?Q9, **. D?(#DD', 0*ril '(DD. 1references3 X. Clerk Ma@-ell, 0 "reatise on Electricit% and Magnetism, Erd ed., vol. ?. 2@+ordC Clarendon, '5(?, **.45#9E. !.S. Xacobs and C. . $ean, UBine *articles, thin +ilms and e@change anisotro*%,V in Magnetism, vol. !!!, ,.". Rado and P. Suhl, Eds. NeYorkC 0cademic, '(4E, **. ?9'#ED). K. Elissa, U"itle o+ *a*er i+ kno-n,V un*ublished. R. Nicole, U"itle o+ *a*er -ith onl% +irst -ord ca*itali<ed,V X. Name Stand. 0bbrev., in *ress. Y. Yoro<u, M. Pirano, K. 2ka, and Y. "aga-a, UElectron s*ectrosco*% studies on magneto#o*tical media and *lastic substrate inter+ace,V !EEE "ransl. X. Magn. Xa*an, vol. ?, **. 9Q)#9Q', 0ugust '(59 :Digests (th 0nnual Con+. Magnetics Xa*an, *. E)', '(5?;. M. Young, "he "echnical 6riter/s Pandbook. Mill Valle%, C0C >niversit% Science, '(5(.