!"#$%&'()*+,-&.

+/)(-&
0+#&1(2"3&&
4/("%&56
-7
8&9:55&
;+,7$'$--"<&=7+&>$2+&>%$?&
@;AB&C&DDBEFFG&
@#3-(H,-)(3<&A(I&4I&4#-)#")&4(()J)8&A(I&D(",&;I&K,7?$(-L&
=43<&="M&;$(-"#8&.J$#&K-+/7+#38&A+N"#&OHP7+38&Q)37&R+$N+(8&K+$#&!(H,7-&
!"#$%&'(&)'*+%*+,&
AbsLracL ......................................................................................................... D(()(S&T))2M$(2&#)-&U+V"#+UI
lnLroducLlon ................................................................................................................................................. 3
SysLem Cvervlew ......................................................................................................................................... 4
LlecLrlcal ComponenLs ................................................................................................................................. 3
Speclal SysLem: Mechanlcs for Llnear AcLuaLors ln a non-Llnear ÞlaLform ................................................. 6
ueslgn 1 ................................................................................................................................................... 7
ueslgn 2 ................................................................................................................................................... 8
ueslgn 3 ................................................................................................................................................... 9
1he Cpposable 1humb ........................................................................................................................... 10
1enslon, lrlcLlon, and 8eslsLance .......................................................................................................... 11
Speclal Sensor: CompuLer vlslon ............................................................................................................... 12
8eferences ................................................................................................................................................. 16
Appendlx A - Ardulno Code ....................................................................................................................... 16
Appendlx 8 - MA1LA8 Code for uemo uay ............................................................................................... 21
Appendlx C - MA1LA8 Code for PSv SlmulLaneous uoL 8ecognlLlon ....................................................... 23
Appendlx u - MA1LA8 Code for 8C8 Analysls parL 1 ................................................................................ 26
Appendlx L - MA1LA8 Code for 8C8 Analysls ÞarL 2 ................................................................................. 27
Appendlx l - MA1LA8 Code for Llnear 8egresslon .................................................................................... 28


llgure 1 - Mohawk ueLecLlon

-*+.'/01+2'*&

MechaÞaLLl: 1he Cake Claw ls a Lhree appendage mechanlcal claw LhaL uses compuLer vlslon Lo mlmlc
hand gesLures much llke Lhe game of paLLy cake. MechaÞaLLl feaLures a complex mechanlcal sLrucLure
LhaL was deslgned Lo be anaLomlcally correcL wlLh respecL Lo human hands, proporLlonally and
mechanlcally. 1he lndlvldual flnger dlglL measuremenLs were based off measuremenLs Laken from a
skeleLal repllca borrowed from Lhe ul anaLomy lab ln Lhe llorlda Cym and scaled Lo be 60° larger Lhan
a Lyplcal human. 1he acLual hand conslsLs of Lhe ma[or appendages necessary for full hand funcLlonallLy:
Lhe Lhumb, lndex, and mlddle flngers. 1he lndex and mlddle flnger each have Lhree lsolaLed degrees of
freedom LhaL allow each [olnL Lo bend lndependenLly of oLher [olnLs ln Lhe flnger. 1he Lhumb has Lwo
degrees of flnger as well as a fully opposable base [olnL. 1he hand ls conLrolled and acLuaLed Lhrow
several elecLronlc componenLs and reacLs Lo oLher hands uslng varlous sensors and compuLer vlslon
lnvolvlng heavy lmage processlng.



llgure 2 - MechaÞaLLl
34,+%5&67%.72%8&










llgure 3 - Ceneral llowcharL

1he prlmary hub for all roboLlc lnLenL ls declded and conLrolled by Lhe Ardulno uuemllanove
mlcroconLroller. 1he LoglLech webcam capLures lmages. 1he lmages are collecLed and processed by
MA1LA8 and hand orlenLaLlons are declded. 1he orlenLaLlons are passed Lo Lhe Ardulno. 1he ardulno
slmulLaneously reads ln daLa from Lhe ulLrasonlc sensor (range flnder). 1he ardulno Lhen compares
values Laken from Lhe range flnder and MA1LA8 before decldlng whaL command Lo send Lo Lhe servo
conLroller. 1he servo conLroller recelves commands serlally from Lhe Ardulno and conLrols Lhe servos
uslng pulse wldLh modulaLlon. 1he servos are connecLed Lo llnear acLuaLors LhaL pass Lhrough some
parLs and [olnLs of MechaÞaLLl Lo apply lsolaLed Lorque Lo each servo's asslgned [olnL.







Þarallax Þlng)))
ulLraSonlc Sensor
LynxMoLlon SSC-32
Servo ConLroller
Ardulno
uuemllanove

MA1LA8 on
Wlndows xÞ
LoglLech
Webcam
PlLec PS-311
Servo
Llnear
AcLuaLor
9$%1+.21"$&)'5:'*%*+,&

Whlle MechaÞaLLl ls hlghly mechanlcal ln deslgn, she also uses qulLe a deal of elecLrlcal componenLs. 1he
parL breakdown ls as follows:
• 1 x LoglLech Webcam
o CapLures lmages and LransmlLs Lhem Lo Lhe ÞC ln 8C8 formaL.
• 1 x ÞC LhaL runs Wlndows xÞ and MA1LA8
o 2 Cb 8AM
o 2 MPz Þrocessor
o SysLem specs llmlL lmage processlng speed.
• 1 x Þarallax Þlng))) ulLrasonlc Sensor
o 8ange llnder
• 1 x Ardulno uuemllanove
o MlcroconLroller
o CenLral lnLelllgence unlL
• 1 x LynxMoLlon SSC-32 Servo ConLroller
o Laslly allows for ÞWM conLrol of mulLlple servos slmulLaneously.
o Serlal CommunlcaLlon, recelves sLrlngs for commands.
• 12 x PlLec PS-311 Servos
o 180¨ of 8oLaLlon
o SLall 1orque of 49 Cz/ln when supplled wlLh 6v.











3:%12"$&34,+%5;&<%1="*21,&('.&>2*%".&?1+0"+'.,&2*&"&@'*A>2*%".&B$"+('.5&


llgure 4 - Mechanlcs lrom vlrLual Lo 8eallLy
1he mechanlcal deslgn of MechaÞaLLl was lnsplred by Lhe human anaLomy and a general undersLandlng
of mechanlcs, Lorques, and pulley sysLems. As an elecLrlcal englneer, l had a severe dlsadvanLage
enLerlng Lhe deslgn phase buL used Lhls opporLunlLy Lo learn a loL abouL mechanlcs lnsLead of
purchaslng a predeslgned/prefabrlcaLed shell Lo acLuaLe. 1he orlglnal deslgn was creaLed over Lwo
weeks and underwenL Lwo ma[or adapLaLlons for a cumulaLlve deslgn Llme of over a monLh. LxpecLed,
slmulaLed Lorque ls very dlfferenL ln pracLlce and several more modlflcaLlons were made afLer Lhe hand
was bullL Lo lncrease funcLlonallLy and reduce Lorque.



C%,2D*&E&
Llnear acLuaLors based on a pulley sysLem were selecLed and deslgned Lo lncrease Lhe reallsm and
relaLlon Lo Lhe Lrue anaLomy of Lhe hand whlch requlres muscle conLracLlons Lo acLuaLe [olnLs.

Cne of Lhe maln lssues was LhaL Lo bend a [olnL, a capable had Lo shorLen and pulllng Lhe cable aLLached
Lo Lhe flnger Llp shorLen and caused a cascaded effecL forclng all oLher [olnLs Lo bend as well. AfLer
repalrlng a broken brake llne on my moLorcycle, l came up wlLh Lhe ldea of a cable/sheaLhe deslgn whlch
would lnhlblL Lhe cable from bendlng ln lnapproprlaLe [olnLs. 1hls worked well yeL caused slgnlflcanL
reslsLanL Lorque on Lhe oLher [olnLs when Lhey had Lo bend Lhe hard sheaLhe.
1he second lssue was locaLlng Lhe approprlaLe polnL Lo afflx Lhe cables Lo maxlmlze Lorque aL Lhe [olnL.
1he [olnLs roLaLed around a cenLral axel and had anoLher pln aLLached parallel lL for Lhe cable Lo connecL
Loo.

llgure 3 - ueslgn 1

C%,2D*&F&
1he second deslgn was heavlly based off Lhe flrsL one. 1he ma[or modlflcaLlon was lnverLlng Lhe
lnLerlocklng [olnLs. ln Lhe flrsL deslgn Lhe [olnLs were creaLed so LhaL Lhey could only roLaLe 90¨ (from
sLralghL Lo benL) and could noL bend backwards or Loo far forwards. 1he orlglnal deslgn was good buL
lnvolved havlng Lhe back of Lhe [olnL, Lhe ºknuckle", exLend down from Lhe hlgher parL of Lhe flnger.
Changlng Lhls [olnL, so LhaL Lhe knuckle rose up from Lhe lower end, allowed a flsh-eye screw Lo be
aLLached aL a hlgher polnL ln Lhe flnger creaLlng more Lorque on Lhe [olnL when lL was conLracLed. 1lm
MarLln's guldance dlrecLed me Lo Lhls modlflcaLlon.

llgure 6 - ueslgn 2

C%,2D*&G&
1he Lhlrd deslgn greaLly resembled Lhe flrsL Lwo buL perfecLed Lhem. 1he hand dlmenslons were
lncreased so Lhe hand was 60° larger Lhan a normal human hand for Lwo reasons. Cne, Lhe 3
dlmenslonal sLrucLure requlred Lwo layers of wood per slde (each 3mm Lhlck) so Lhe deslgn was
naLurally bulky. 1wo, Lhe lncreased slze allowed more space for cables Lo be aLLached and ad[usLed lf
someLhlng needed Lo be changed ln Lhe fuLure.
8esldes lncreaslng Lhe dlmenslons, Lhe pulley sysLem was changed from bralded sLeel flex cable and
hard plasLlc sheaLhes Lo flshlng wlre and rubber Lublng. 1he flshlng wlre reduced space requlremenLs
lnslde Lhe flnger. AfLer reallzlng Lhe Lorque llmlLaLlons of Lhe moLors lL was obvlous LhaL 10 lb LesL llne
would noL break before Lhe moLor burnL ouL and Lherefore would be a beneflclal replacemenL Lo Lhe
Lhlck meLal cables. 1he rubber Lublng also worked well for Lhe angle [olnLs as lL was sLlff enough Lo noL
be benL by Lhe wlre yeL sofL enough Lo bend wlLhouL addlng exLra Lorque on Lhe [olnLs Lhey passed
Lhrough.

llgure 7 - ueslgn 3

!=%&6::',"#$%&!=05#&
1he Lhumb was an excepLlonally fun and challenglng parL of Lhe deslgn process. 1o remaln anaLomlcally
correcL, l chose a ball and sockeL Lype [olnL. 1he [olnL conslsLed of a 3/8
Lh
ln Lhreaded rod exLendlng from
Lhe upper parL of Lhe Lhumb. 1he rod was LermlnaLed wlLh an acorn nuL whlch connecLed smooLhely
wlLh a flnnlsh washer glue Lo Lhe base of Lhe hand. 1he mosL Lroublesome parL of Lhls was LhaL Lhere
was no lnLerlocklng parLs ln Lhe [olnL. lnlLlally l planned Lo [usL aLLach rubber bands around lL as Lhe [olnL
would need counLer forces Lo have a ºreLurn Lo zero" poslLlon llke Lhe normal [olnLs buL found LhaL exLra
supporL had Lo be found elsewhere. 1he flnal deslgn lnvolved a Lendon & llgamenL Lype comblnaLlon ln
whlch elasLlc nylon cord was used Lo LlghLen and secure Lhe ball lnslde Lhe sockeL whlle an ouLer
surroundlng layer of rubber bands also held Lhe [olnL connecLed and allowed for a wlder range of sLaLlc
poslLlonlng.

llgure 8 - 1humb and lorearm ueslgns

!%*,2'*H&I.21+2'*H&"*/&J%,2,+"*1%&
1here were several non-ldeallLles LhaL conLrlbuLed Lo Lhe complexlLy of Lhe pro[ecL. labrlcaLlng Lhe hand
ouL of º2-u" wood requlred a more lnLelllgenL deslgn of lnLerlocklng parLs and layered sldes. Lach [olnL
roLaLed around wooden dowel rods and wood on wood ls noL an ldeal surface conLacL for slldlng parLs.
Wu40 worked as a decenL lubrlcanL buL Lhe besL one was an oll based lubrlcanL made by 3lnCne.
1he pulley cable sysLem lnvolved slngle dlrecLlon Lenslon of Lhe acLuaLors and Lherefor Lhe flngers
needed and ºreLurn Lo zero" reverse acLuaLor or slmply sLaLed, some Lype of elasLlc counLer force Lo
reLracL Lhe [olnL afLer lL was noL longer belng flexed. nylon cables wore ouL fasL, rubber bands worked
well buL had Lo be sLreLched along Lhe back as Lhe reslsLlve force ls proporLlonal Lo Lhe LoLal maLerlal
lengLh vs Lhe lengLh lL ls belng sLreLched (shorL rubber bands don'L sLreLch as well over a one lnch
dlsLance as long rubber bands do).
Sprlngs were also used by Lhey made a dlsLurblng cllnk sound when Lhey grlnded agalnsL corners and
also losL shape and deformed.















3:%12"$&3%*,'.;&)'5:0+%.&K2,2'*&


llgure 9 - MechaÞaLLl. whaL choo lookln aL?!
1o play paLLy cake, MechaÞaLLl needed Lo be able Lo do Lwo Lhlngs: recognlze oLher hands and recognlze
Lhe hand orlenLaLlon or poslLlon characLerlsLlcs. 1he flrsL parL was lmplemenLed based on a raLher
sLupldly slmple reallzaLlon/assumpLlon: a hand cannoL be presenL wlLhouL a person, a person ls bulky, lf
a range flnder senses a dlsLance change from lnflnlLy Lo a flnlLe value some ob[ecL has appeared.
1herefore lf MechaÞaLLl deLecLs a closeby ob[ecL lL assumes lL's a person and sLarLs uslng Lhe webcam Lo
recognlze Lhe hand LhaL probably now exlsLs.
1he hand orlenLaLlon characLerlsLlcs were ldenLlfled based on lmage processlng. 1he lmage processlng
was done ln MA1LA8 and lnvolved hue-saLuraLlon-value analysls, blob deLecLlon, red-green-blue
analysls, Lhresholdlng, and masklng. 1he Lechnlques requlred ln lmage processlng were unknown Lo me
prlor Lo Lhls class and were very fun Lo experlmenL wlLh durlng Lhe whole learnlng and deslgn process.
Pand poslLlons were calculaLed based on doL and llne analysls where each flnger had a unlque color
assoclaLed wlLh lL. Lach [olnL had a doL assoclaLed wlLh lL so LhaL when an lmage was examlned lf Lhree
red doLs were presenL Lhe lndex flnger was fully exLended buL lf Lhe flnger was benL some doLs would be
hldden and only one or Lwo mlghL be vlslble so MechaÞaLLl would know Lo bend parL of Lhe lndex flnger
(or Mlddle flnger lf blue was mlsslng).
1he Lhumb orlenLaLlon was sllghLly more complex as Lhe bendlng and shlfLlng would noL resulL ln hldlng
doLs. 1hls was solved wlLh angle measuremenLs. 1he palm had one llne drawn on lL whlle a second llne
wenL along Lhe prlmary llne of Lhe Lhumb so LhaL lf Lhe Lhumb was shlfLed Loward Lhe palm Lhe angle
beLween Lhe Lwo llnes would be reduced. Llne and angle measuremenLs were calculaLed by masklng
lndlvldual llnes ln Lhe lmage, ldenLlfylng x and ? coordlnaLes for llne plxel values and dolng llnear
regresslon on Lhe plxel coordlnaLes Lo creaLe a characLerlsLlc llne. 1he angle beLween Lhe Lwo llnes was
Lhen calculaLed uslng Lhe coslne funcLlon.
8elow are several plcLures LhaL show Lhe sLages of PSv/8C8 analysls, Lhresholdlng, blob deLecLlon, and
llnear regresslon.


llgure 10 - 1heoreLlcal Mechavlslon




llgure 11 - AcLual Mechavlslon - 1hls ls Lhe ouLpuL ploLs from MechaÞaLLl ldenLlfylng red doLs for Lhe
lndex flnger.




llgure 12 - AcLual Mechavlslon - 1hls ls Lhe ouLpuL ploLs from MechaÞaLLl ldenLlfylng Lhumb poslLlons,
noLlce Lhe boLLom ploLs ln whlch Lhe red llne ls drawn based on Lhe llnear regresslon.










J%(%.%*1%,&
&
K"M/%+&>)%)(&A+-+,-")#&WJ&OH+
Lxample code LhaL shows Lhe baslcs of hue-saLuraLlon-value lmage processlng. WrlLLen by lmage
AnalysL.
hLLp://www.maLhworks.com/maLlabcenLral/flleexchange/28312-slmple-color-deLecLlon-by-hue
T%)W3A+M)
Lxample code LhaL shows Lhe baslcs of blob deLecLlon, counLlng, and ellmlnaLlng. WrlLLen by lmage
AnalysL.
hLLp://www.maLhworks.com/maLlabcenLral/flleexchange/23137


?::%*/2L&?&M&?./02*'&)'/%&

lnL counL = 0,
lnL lncomlng8yLe = 0,
consL lnL plngÞln = 8,

vold seLup()
[
Serlal.begln(9600),
}

vold loop()
[
whlle (CheckulsLCm() < 80)
[ lf(Serlal.avallable() > 0)
[
lncomlng8yLe = Serlal.read(),

lf(lncomlng8yLe == 48)
[
8elaxlndex(),
8elaxMlddle(),
}
lf(lncomlng8yLe == 49)
[
lndex1(),
8elaxMlddle(),
}
lf(lncomlng8yLe == 30)
[
lndex123(),
8elaxMlddle(),
}
lf(lncomlng8yLe == 31)
[
8elaxlndex(),
Mlddle1(),
}
lf(lncomlng8yLe == 32)
[
lndex1(),
Mlddle1(),
}
lf(lncomlng8yLe == 33)
[
lndex123(),
Mlddle1(),
}
lf(lncomlng8yLe == 34)
[
8elaxlndex(),
Mlddle123(),
}
lf(lncomlng8yLe == 33)
[
lndex1(),
Mlddle123(),
}
lf(lncomlng8yLe == 36)
[
lndex123(),
Mlddle123(),
}
lf(lncomlng8yLe == 37)
[
lndex123(),
Mlddle123(),
}
lf(lncomlng8yLe == 38)
[
1humbneuLral(),
}
lf(lncomlng8yLe == 39)
[
1humbWlde(),
}
lf(lncomlng8yLe == 60)
[
1humbClose(),
}
}
}

lf (counL == 0)
[ 8elaxAll(), }
lf (counL == 1)
[ lndex1(), Mlddle1(), }
lf (counL == 2)
[ lndex2(), Mlddle2(), }
lf (counL == 3)
[ lndex3(), Mlddle3(), }
lf (counL == 4)
[ 1humbneuLral(), }
lf (counL == 3)
[ 1humbWlde(), }
lf (counL == 6)
[ 1humbClose(), }
lf (counL == 7)
[ 1humblC(), }
lf (counL == 8)
[ 1humb8C(), }


lf (counL > 8)
[ counL = 0, }

delay(1000),
counL++,
}

vold 8elaxlndex()
[
Serlal.prlnLln("#20Þ300 #21Þ300 #22Þ300 11000 <cr>"),
}
vold 8elaxMlddle()
[
Serlal.prlnLln("#28Þ300 #29Þ300 #30Þ300 11000 <cr>"),
}
vold 8elax1humb()
[
Serlal.prlnLln("#1Þ730 #2Þ730 #4Þ730 #3Þ730 #8Þ730 #9Þ730 11000 <cr>"),
}
vold 8elaxAll()
[
Serlal.prlnLln("#20Þ300 #21Þ300 #22Þ300 #28Þ300 #29Þ300 #30Þ300 #1Þ730 #2Þ730 #4Þ730 #3Þ730
#8Þ730 #9Þ730 11000 <cr>"),
}

vold lndex1()
[
Serlal.prlnLln("#20Þ2000 #21Þ730 #22Þ730 11000 <cr>"),
}
vold lndex12()
[
Serlal.prlnLln("#20Þ2000 #21Þ2000 #22Þ730 11000 <cr>"),
}
vold lndex123()
[
Serlal.prlnLln("#20Þ2000 #21Þ2000 #22Þ2000 11000 <cr>"),
}
vold lndex2()
[
Serlal.prlnLln("#20Þ730 #21Þ2000 #22Þ730 11000 <cr>"),
}
vold lndex23()
[
Serlal.prlnLln("#20Þ730 #21Þ2000 #22Þ2000 11000 <cr>"),
}
vold lndex3()
[
Serlal.prlnLln("#20Þ730 #21Þ730 #22Þ2000 11000 <cr>"),
}


vold Mlddle1()
[
Serlal.prlnLln("#28Þ2000 #29Þ730 #30Þ730 11000 <cr>"),
}
vold Mlddle12()
[
Serlal.prlnLln("#28Þ2000 #29Þ2000 #30Þ730 11000 <cr>"),
}
vold Mlddle123()
[
Serlal.prlnLln("#28Þ2000 #29Þ2000 #30Þ2000 11000 <cr>"),
}
vold Mlddle2()
[
Serlal.prlnLln("#28Þ730 #29Þ2000 #30Þ730 11000 <cr>"),
}
vold Mlddle23()
[
Serlal.prlnLln("#28Þ730 #29Þ2000 #30Þ2000 11000 <cr>"),
}
vold Mlddle3()
[
Serlal.prlnLln("#28Þ730 #29Þ730 #30Þ2000 11000 <cr>"),
}

vold 1humbneuLral()
[
Serlal.prlnLln("#1Þ730 #2Þ730 #4Þ730 #3Þ730 #9Þ730 #10Þ730 11000 <cr>"),
}

vold 1humbWlde()
[
Serlal.prlnLln("#1Þ730 #2Þ1300 #4Þ730 #3Þ730 11000 <cr>"),
}

vold 1humbClose()
[
Serlal.prlnLln("#1Þ1300 #2Þ730 #4Þ730 #3Þ730 11000 <cr>"),
}

vold 1humblC()
[
Serlal.prlnLln("#1Þ730 #2Þ1300 #4Þ1300 #3Þ730 11000 <cr>"),
}

vold 1humb8C()
[
Serlal.prlnLln("#1Þ730 #2Þ1300 #4Þ730 #3Þ1300 11000 <cr>"),
}


long CheckulsLCm()
[
long mlcroseconds, cenLlmeLers,

plnMode(plngÞln, Cu1Þu1),
dlglLalWrlLe(plngÞln, LCW),
delayMlcroseconds(2),
dlglLalWrlLe(plngÞln, PlCP),
delayMlcroseconds(3),
dlglLalWrlLe(plngÞln, LCW),

plnMode(plngÞln, lnÞu1),
mlcroseconds = pulseln(plngÞln, PlCP),

reLurn cenLlmeLers = mlcroseconds / 29 / 2,
}

?::%*/2L&N&M&<?!>?N&)'/%&('.&C%5'&C"4&

tic; % Start timer.

%%%%% CLEAN MATLAB %%%%%
clear;
clc;
close all;

disp('Initializing Vision...');
Occulus=videoinput('winvideo',1,'RGB24_640x480');

disp('Initializing Mecha-Patti...');
obj = serial('COM14');
set(obj, 'Terminator', 'CR');
set (obj, 'BaudRate',9600,'DataBits',8,'Parity','none','StopBits',1)
obj.FlowControl = 'none';
fopen(obj)
disp('Initialization Complete.');

while(1)
%i = 1:20
tic;

% close all;
% pause(2);
rgbImage = getsnapshot(Occulus);
%rgbImage = imread('pic10.jpg');
% imshow(rgbImage); drawnow;
% close all;
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%HSV - Hue, Saturation, Value Analysis%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
hsvImage = rgb2hsv(rgbImage); %Convert RGB image into HSV image
hImage = hsvImage(:,:,1); %Isolate hue layer
sImage = hsvImage(:,:,2); %Isolate saturation layer
vImage = hsvImage(:,:,3); %Isolate value layer

BlueHueLow = 0.6; %Set threshold levels
BlueHueHigh = 0.8;
BlueSatLow = 0.5;
BlueSatHigh = 1;
BlueValLow = 0.2;
BlueValHigh = 1;

BlueHueMask = (hImage >= BlueHueLow) & (hImage <= BlueHueHigh);
BlueSatMask = (sImage >= BlueSatLow) & (sImage <= BlueSatHigh);
BlueValMask = (vImage >= BlueValLow) & (vImage <= BlueValHigh);
BlueHSVMask = uint8(BlueHueMask & BlueSatMask & BlueValMask);
smallestAcceptableArea = 100;
BlueHSVMask = uint8(bwareaopen(BlueHSVMask, smallestAcceptableArea));
structuringElement = strel('disk', 4);
BlueHSVMask = imclose(BlueHSVMask, structuringElement);
InvBlueHSVMask = ~BlueHSVMask;
BlueHSVMask = cast(BlueHSVMask, class(rgbImage));
BlueMaskedImageR = BlueHSVMask .* rgbImage(:,:,1);
BlueMaskedImageG = BlueHSVMask .* rgbImage(:,:,2);
BlueMaskedImageB = BlueHSVMask .* rgbImage(:,:,3);
BlueMaskedRGBImage = cat(3, BlueMaskedImageR, BlueMaskedImageG,
BlueMaskedImageB);
InvBlueHSVMask = cast(InvBlueHSVMask, class(rgbImage));
BlueMaskedImageR = InvBlueHSVMask .* rgbImage(:,:,1);
BlueMaskedImageG = InvBlueHSVMask .* rgbImage(:,:,2);
BlueMaskedImageB = InvBlueHSVMask .* rgbImage(:,:,3);
BlueMaskedRGBImage2 = cat(3, BlueMaskedImageR, BlueMaskedImageG,
BlueMaskedImageB);

RedHueLow = 0.9; %Set threshold levels
RedHueHigh = 1;
RedSatLow = 0.7;
RedSatHigh = 1;
RedValLow = 0;
RedValHigh = 1;

RedHueMask = (hImage >= RedHueLow) & (hImage <= RedHueHigh);
RedSatMask = (sImage >= RedSatLow) & (sImage <= RedSatHigh);
RedValMask = (vImage >= RedValLow) & (vImage <= RedValHigh);
RedHSVMask = uint8(RedHueMask & RedSatMask & RedValMask);
smallestAcceptableArea = 100;
RedHSVMask = uint8(bwareaopen(RedHSVMask, smallestAcceptableArea));
structuringElement = strel('disk', 4);
RedHSVMask = imclose(RedHSVMask, structuringElement);
InvRedHSVMask = ~RedHSVMask;
RedHSVMask = cast(RedHSVMask, class(rgbImage));
RedMaskedImageR = RedHSVMask .* rgbImage(:,:,1);
RedMaskedImageG = RedHSVMask .* rgbImage(:,:,2);
RedMaskedImageB = RedHSVMask .* rgbImage(:,:,3);
RedMaskedRGBImage = cat(3, RedMaskedImageR, RedMaskedImageG,
RedMaskedImageB);
InvRedHSVMask = cast(InvRedHSVMask, class(rgbImage));
RedMaskedImageR = InvRedHSVMask .* rgbImage(:,:,1);
RedMaskedImageG = InvRedHSVMask .* rgbImage(:,:,2);
RedMaskedImageB = InvRedHSVMask .* rgbImage(:,:,3);
RedMaskedRGBImage2 = cat(3, RedMaskedImageR, RedMaskedImageG,
RedMaskedImageB);

YellowHueLow = 0; %Set threshold levels
YellowHueHigh = 0.4;
YellowSatLow = 0.5;
YellowSatHigh = 1;
YellowValLow = 0;
YellowValHigh = 1;

YellowHueMask = (hImage >= YellowHueLow) & (hImage <= YellowHueHigh);
YellowSatMask = (sImage >= YellowSatLow) & (sImage <= YellowSatHigh);
YellowValMask = (vImage >= YellowValLow) & (vImage <= YellowValHigh);
YellowHSVMask = uint8(YellowHueMask & YellowSatMask & YellowValMask);
smallestAcceptableArea = 100;
YellowHSVMask = uint8(bwareaopen(YellowHSVMask, smallestAcceptableArea));
structuringElement = strel('disk', 4);
YellowHSVMask = imclose(YellowHSVMask, structuringElement);
InvYellowHSVMask = ~YellowHSVMask;
YellowHSVMask = cast(YellowHSVMask, class(rgbImage));
YellowMaskedImageR = YellowHSVMask .* rgbImage(:,:,1);
YellowMaskedImageG = YellowHSVMask .* rgbImage(:,:,2);
YellowMaskedImageB = YellowHSVMask .* rgbImage(:,:,3);
YellowMaskedRGBImage = cat(3, YellowMaskedImageR, YellowMaskedImageG,
YellowMaskedImageB);
InvYellowHSVMask = cast(InvYellowHSVMask, class(rgbImage));
YellowMaskedImageR = InvYellowHSVMask .* rgbImage(:,:,1);
YellowMaskedImageG = InvYellowHSVMask .* rgbImage(:,:,2);
YellowMaskedImageB = InvYellowHSVMask .* rgbImage(:,:,3);
YellowMaskedRGBImage2 = cat(3, YellowMaskedImageR, YellowMaskedImageG,
YellowMaskedImageB);

%%%%% Count Dots %%%%%
RedImage = RedMaskedRGBImage(:,:,1);
BWImage = im2bw(RedImage,0.1);
BW_filled_Red = imfill(BWImage,'holes');
boundaries = bwboundaries(BW_filled_Red);
size(boundaries);
[redL, numRED] = BWLABEL(BW_filled_Red);

BlueImage = BlueMaskedRGBImage(:,:,3);
BlueBWImage = im2bw(BlueImage,0.1);
BW_filled_Blue = imfill(BlueBWImage,'holes');
boundaries = bwboundaries(BW_filled_Blue);
size(boundaries);
[blueL, numBLUE] = BWLABEL(BW_filled_Blue);

%%%%% Plot Images %%%%%
close all;
figure; set(gcf, 'Position', get(0, 'ScreenSize'));
subplot(3,3,1); imshow(rgbImage);
subplot(3,3,4); imshow(RedMaskedRGBImage);
subplot(3,3,5); imshow(YellowMaskedRGBImage);
subplot(3,3,6); imshow(BlueMaskedRGBImage);
subplot(3,3,7); imshow(BW_filled_Red);
subplot(3,3,8); imshow(BW_filled_Blue);
pause(3);

%%%%% Results %%%%%

numRED
numBLUE

if(numRED >= 3 & numBLUE >= 3)
fwrite(obj,48);
elseif (numRED == 1 & numBLUE >= 3)
fwrite(obj,49);
elseif (numRED == 0 & numBLUE >= 3)
fwrite(obj,50);
elseif (numRED >= 3 & numBLUE == 1)
fwrite(obj,51);
elseif (numRED == 1 & numBLUE == 1)
fwrite(obj,52);
elseif (numRED == 0 & numBLUE == 1)
fwrite(obj,53);
elseif (numRED >= 3 & numBLUE == 0)
fwrite(obj,54);
elseif (numRED == 1 & numBLUE == 0)
fwrite(obj,55);
elseif (numRED == 0 & numBLUE == 0)
fwrite(obj,56);
% elseif (numRED == 0 & numBLUE == 0)
% fwrite(obj,57);
end

toc
end

fclose(obj);

disp('Sequence Complete');
?::%*/2L&)&M&<?!>?N&)'/%&('.&O3K&3250$+"*%'0,&C'+&J%1'D*2+2'*&

%clear; clc; close all;
disp('Running MP_Limbo.m');
disp('This program uses HSV analysis and thresholding in conjunction with RGB
masking for color detection.');

%Read In Image
rgbImage = imread('pic2.jpg');
[rows columns numberOfColorBands] = size(rgbImage);

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%HSV - Hue, Saturation, Value Analysis%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
hsvImage = rgb2hsv(rgbImage); %Convert RGB image into HSV image
hImage = hsvImage(:,:,1); %Isolate hue layer
sImage = hsvImage(:,:,2); %Isolate saturation layer
vImage = hsvImage(:,:,3); %Isolate value layer

hueThresholdLow = 0; %Set threshold levels
hueThresholdHigh = 0.3;
saturationThresholdLow = 0;
saturationThresholdHigh = 0.5;
valueThresholdLow = 0.2;
valueThresholdHigh = 1;
%Apply thresholds
hueMask = (hImage >= hueThresholdLow) & (hImage <= hueThresholdHigh);
saturationMask = (sImage >= saturationThresholdLow) & (sImage <=
saturationThresholdHigh);
valueMask = (vImage >= valueThresholdLow) & (vImage <=
valueThresholdHigh);
%Create mask of HSV layers
hsvMask = uint8(hueMask & saturationMask & valueMask);
%Eliminate small pixel groups(objects)
smallestAcceptableArea = 10;
hsvMask = uint8(bwareaopen(hsvMask, smallestAcceptableArea));
%Smoothe Edges
structuringElement = strel('disk', 4);
hsvMask = imclose(hsvMask, structuringElement);
InvHsvMask = ~hsvMask;


%Convert mask to proper data type.
hsvMask = cast(hsvMask, class(rgbImage));
%Apply the hsvMask to individual RGB
layers.
maskedImageR = hsvMask .* rgbImage(:,:,1);
maskedImageG = hsvMask .* rgbImage(:,:,2);
maskedImageB = hsvMask .* rgbImage(:,:,3);
%Combine layers to 1 RGB image.
maskedRGBImage = cat(3, maskedImageR, maskedImageG, maskedImageB);

InvHsvMask = cast(InvHsvMask, class(rgbImage));
%Inverted HSV Mask
%Apply the InvHsvMask to individual RGB
layers.
maskedImageR = InvHsvMask .* rgbImage(:,:,1);
maskedImageG = InvHsvMask .* rgbImage(:,:,2);
maskedImageB = InvHsvMask .* rgbImage(:,:,3);
%Combine layers to 1 RGB image.
maskedRGBImage2 = cat(3, maskedImageR, maskedImageG, maskedImageB);

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%PLOT RESULTS%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

figure; set(gcf, 'Position', get(0, 'ScreenSize')); %Create a new full
size image.

subplot(2,1,1); imshow(rgbImage); %Input
subplot(2,1,2); imshow(maskedRGBImage2); %Output

disp('Program Complete.');

?::%*/2L&C&M&<?!>?N&)'/%&('.&JPN&?*"$4,2,&:".+&E&

%This program will take the masked image from MP_limbo.m and isolate color
dots
%for blob detection.

ColoredImage = maskedRGBImage2;

%ColoredImage = rgb2hsv(ColoredImage);
%RedImage(:,:,2) = 0;
%RedImage(:,:,3) = 0;

[x y z] = size(ColoredImage);

rgbXtreme = ColoredImage;
for i = 1:x
for j = 1:y
PixelVals = [rgbXtreme(i,j,1) rgbXtreme(i,j,2) rgbXtreme(i,j,3)];
[grb, rgbDepth] = find(PixelVals == max(PixelVals));
if rgbDepth == 1
rgbXtreme(i,j,2) = 0;
rgbXtreme(i,j,3) = 0;
end
if rgbDepth == 2
rgbXtreme(i,j,1) = 0;
rgbXtreme(i,j,3) = 0;
end
if rgbDepth == 3
rgbXtreme(i,j,1) = 0;
rgbXtreme(i,j,2) = 0;
end
end
end

%rgbXtreme2 = ColoredImage;
%for i = 1:x
% for j = 1:y
% PixelVals = [rgbXtreme(i,j,1) rgbXtreme(i,j,2) rgbXtreme(i,j,3)];
% [grb, rgbMax] = find(PixelVals == max(PixelVals));
% [grb, rgbMean] = find(PixelVals == mean(PixelVals));
% [grb, rgbMin] = find(PixelVals == min(PixelVals));
% rgbXtreme2(i,j,rgbMax) = rgbXtreme2(i,j,rgbMax) -
rgbXtreme2(i,j,rgbMean);
% rgbXtreme2(i,j,rgbMean) = 0;
% rgbXtreme2(i,j,rgbMin) = 0;
% end
%end

RedPic = ((rgbXtreme(:,:,1) >= 140) & (rgbXtreme(:,:,1) <= 160));


figure;
subplot(3,1,1); imshow(ColoredImage);
subplot(3,1,2); imshow(rgbXtreme);
%subplot(3,1,3); imshow(rgbXtreme2);


?::%*/2L&9&M&<?!>?N&)'/%&('.&JPN&?*"$4,2,&B".+&F&&

ColoredImage = maskedRGBImage2;

Red = ColoredImage(:,:,1);
Green = ColoredImage(:,:,2);
Blue = ColoredImage(:,:,3);
Null = zeros(size(Red));

Avg = Red.*Green.*Blue;
thresholdValue = 10;
binaryImage = Avg > thresholdValue;
binaryImage = imfill(binaryImage, 'holes');
labeledImage = bwlabel(binaryImage, 8);
blobMeasurements = regionprops(labeledImage, Avg, 'all');
numberOfBlobs = size(blobMeasurements, 1);
allBlobAreas = [blobMeasurements.Area];
allowableAreaIndexes = allBlobAreas > 5000;
keeperIndexes = find(allowableAreaIndexes);
keeperBlobsImage = ismember(labeledImage, keeperIndexes);
TruAvg = Avg; % Simply a copy at first.
TruAvg(~keeperBlobsImage) = 0; % Set all non-keeper pixels to zero.
TruAvg = ~TruAvg;

NewRed = (Red >= 120); NewRed = NewRed .* TruAvg;
NewGreen = (Green >= 120); NewGreen = NewGreen .* TruAvg;
NewBlue = (Blue >= 120); NewBlue = NewBlue .* TruAvg;
NewRed = NewRed .* (~NewGreen);

SuperRed = cat(3, NewRed, Null, Null);
SuperGreen = cat(3, Null, NewGreen, Null);
SuperBlue = cat(3, Null, Null, NewBlue);

% figure;
% subplot(4,3,1); imshow(ColoredImage);
% subplot(4,3,3); imshow(TruAvg);
%
% subplot(4,3,4); imshow(Red);
% subplot(4,3,5); imshow(Green);
% subplot(4,3,6); imshow(Blue);
%
% subplot(4,3,7); imshow(NewRed);
% subplot(4,3,8); imshow(NewGreen);
% subplot(4,3,9); imshow(NewBlue);
%
% subplot(4,3,10); imshow(SuperRed);
% subplot(4,3,11); imshow(SuperGreen);
% subplot(4,3,12); imshow(SuperBlue);


?::%*/2L&I&M&<?!>?N&)'/%&('.&>2*%".&J%D.%,,2'*&

GreenImage = SuperGreen(:,:,2);
BWImage = im2bw(GreenImage,0.5);
BW_filled = imfill(BWImage,'holes');
boundaries = bwboundaries(BW_filled);
size(boundaries);
[L, NUM] = BWLABEL(BW_filled);
stats = regionprops(L,'area','centroid');
allBlobAreas = [stats.Area];
Line1 = (allBlobAreas > 1400 & allBlobAreas < 2500);
Line2 = (allBlobAreas > 2500);
keeperIndexes = find(Line1);
keeperIndexes2 = find(Line2);
GreenLine1 = ismember(L, keeperIndexes);
GreenLine2 = ismember(L, keeperIndexes2);



[L2, numGreen1] = BWLABEL(GreenLine1);
[L2, numGreen2] = BWLABEL(GreenLine2);

%set(gca,'YDir','normal');
[row,col] = find(GreenLine1);
p = polyfit(col,row,1);
pp = polyval(p,1:640);

[row2,col2] = find(GreenLine2);
p2 = polyfit(col2,row2,1);
pp2 = polyval(p2,1:640);


figure;
subplot(2,2,1); imshow(GreenLine1); axis([0 640 0 480]);
subplot(2,2,2); imshow(GreenLine2); axis([0 640 0 480]);
subplot(2,2,3); plot(col,row,'.'); axis([0 640 0 480]); hold on;
plot(1:640,pp,'-r'); hold off;
subplot(2,2,4); plot(col2,row2,'.'); axis([0 640 0 480]); hold on;
plot(1:640,pp2,'-r'); hold off;

Sign up to vote on this title
UsefulNot useful