Technical Articles and Newsletters
Deep Learning for Computer Vision with MATLAB
ByAvinashNehemiahandValerieLeung,MathWorks
Computervisionengineershaveusedmachinelearningtechniquesfordecadestodetectobjectsofinterestinimagesandtoclassifyoridentifycategoriesof
[Link],regions,orobjectsofinterestandthenusethosefeaturestotrainamodeltoclassifyorlearnpatternsinthe
imagedata.
Intraditionalmachinelearning,[Link]
moreimageprocessingoperations,suchascalculatinggradienttoextractthediscriminativeinformationfromeachimage.
[Link],representations,andtasksdirectlyfromimages,text,andsound,eliminatingtheneedfor
manualfeatureselection.
Usingasimpleobjectdetectionandrecognitionexample,thisarticleillustrateshoweasyitistouseMATLABfordeeplearning,evenwithoutextensive
knowledgeofadvancedcomputervisionalgorithmsorneuralnetworks.
Thecodeusedinthisexampleisavailablefordownload.
Getting Started
[Link]
network(CNN),aspecifictypeofdeeplearningalgorithmthatcanbothperformclassificationandextractfeaturesfromrawimages.
TobuildtheobjectdetectionandrecognitionalgorithminMATLAB,[Link]
discriminativefeaturesfromtheimages,andthenuseaMATLABapptotrainamachinelearningalgorithmtodiscriminatebetweencatsanddogs.
Importing a CNN Classifier
WebeginbydownloadingaCNNclassifierpretrainedonImageNet,[Link]
thisexamplewellbeusingtheAlexNetarchitecture.
websave('\networks\[Link]',...
'[Link]
[Link]');
WeimportthenetworkintoMATLABasaSeriesNetworkusingNeuralNetworkToolbox,[Link] SeriesNetwork object
representstheCNN.
%LoadMatConvNetnetworkintoaSeriesNetwork
convnet=helperImportMatConvNet(cnnFullMatFile);
%ViewtheCNNarchitecture
[Link]
Wevestoredtheimagesinseparate cat and dog foldersunderaparentcalled pet_images .TheadvantageofusingthisfolderstructureisthattheMATLAB
imageDatastore wecreatewillbeabletoautomaticallyreadandmanageimagelocationsandclasslabels.( imageDatastore isarepositoryforcollectionsof
datathataretoolargetofitinmemory.)
Weinitializean imageDatastore toaccesstheimagesinMATLAB.
%%Setupimagedata
dataFolder='\data\PetImages';
categories={'Cat','Dog'};
imds=imageDatastore(fullfile(dataFolder,categories),...
'LabelSource','foldernames');
Wethenselectasubsetofthedatathatgivesusanequalnumberofdogandcatimages.
tbl=countEachLabel(imds)
%%Usethesmallestoverlapset
minSetCount=min(tbl{:,2});
%UsesplitEachLabelmethodtotrimtheset.
imds=splitEachLabel(imds,minSetCount,'randomize');
%Noticethateachsetnowhasexactlythesamenumberofimages.
countEachLabel(imds)
Sincethe AlexNet networkwastrainedon227x227pixelimages,[Link]
Technical Articles and Newsletters
toreadandprocessimagesfromthe imageDatastore atthesametime.
%%PreprocessImagesForCNN
%SettheImageDatastoreReadFcn
[Link]=@(filename)readAndPreprocessImage(filename);
%%Dividedataintotrainingandtestingsets
[trainingSet,testSet]=splitEachLabel(imds,0.3,'randomize');
Weusethe readAndPreprocessImage functiontoresizetheimagesto227x227pixels.
functionIout=readAndPreprocessImage(filename)
I=imread(filename);
%Someimagesmaybegrayscale.Replicatetheimage3timesto
%createanRGBimage.
ifismatrix(I)
I=cat(3,I,I,I);
end
%ResizetheimageasrequiredfortheCNN.
Iout=imresize(I,[227227]);
end
Performing Feature Extraction
Wewanttousethisnewdatasetwiththepretrained AlexNet [Link]
adifferentprobleminourcase,classifyingcatsanddogs(Figure1).
[Link].
WepassthetrainingdatathroughtheCNNandusethe activations [Link],
CNNsareformedusinginterconnectedlayersofnonlinearprocessingelements,[Link],and
hiddenlayersprovidenonlinearcomplexitythatgivesaneuralnetworkitscomputationalcapacity.
WhileeachlayerofaCNNproducesaresponsetoaninputimage,[Link]
[Link].
Thelayersatthebeginningofthenetworkcapturebasicimagefeatures,[Link],wevisualizethenetworkfilterweightsfromthe
firstconvolutionallayer(Figure2).
%Getthenetworkweightsforthesecondconvolutionallayer
w1=[Link](2).Weights;
%Scaleandresizetheweightsforvisualization
w1=mat2gray(w1);
w1=imresize(w1,5);
%Displayamontageofnetworkweights.Thereare96individual
%setsofweightsinthefirstlayer.
figure
montage(w1)
title('Firstconvolutionallayerweights')
Technical Articles and Newsletters
[Link].
[Link]"primitive"featuresarethenprocessedbydeepernetwork
layers,[Link]
[Link] activations
method.
[Link].
featureLayer='fc7';
trainingFeatures=activations(convnet,trainingSet,featureLayer,...
'MiniBatchSize',32,'OutputAs','columns');
Training an SVM Classifier Using the Extracted Features
Werenowreadytotraina"shallow"classifierwiththefeaturesextractedinthepreviousstep.Notethattheoriginalnetworkwastrainedtoclassify1000object
[Link].
TheClassificationLearnerappinStatisticsandMachineLearningToolboxletsustrainandcomparemultiplemodelsinteractively(Figure3).
[Link].
Alternatively,wecouldtraintheclassifierinourMATLABscript.
Wesplitthedataintotwosets,[Link],wetrainasupportvectormachine(SVM)classifierusingtheextractedfeaturesby
Technical Articles and Newsletters
callingthe fitcsvm functionusing trainingFeatures astheinputorpredictorsand trainingLabels [Link]
theclassifieronthetestdatatodetermineitsvalidationaccuracy,anunbiasedestimateofhowtheclassifierwouldperformonnewdata.
%%Trainaclassifierusingextractedfeatures
trainingLabels=[Link];
%HereItrainalinearsupportvectormachine(SVM)classifier.
svmmdl=fitcsvm(trainingFeatures,trainingLabels);
%Performcrossvalidationandcheckaccuracy
cvmdl=crossval(svmmdl,'KFold',10);
fprintf('kFoldCVaccuracy:%2.2f\n',[Link])
Wecannowusethe svmmdl classifiertoclassifyanimageasacatoradog(Figure4).
[Link].
Performing Object Detection
Inmostimagesandvideoframes,[Link],inadditiontoadog,therecouldbeatree,oraflockofpigeons,oraraccoonchasingthe
[Link],croptheobject,andthenfeedittotheclassifierinotherwords,if
wecanperformobjectdetection.
Forobjectdetectionwewilluseatechniquecalledopticalflow,whichusesthemotionofpixelsinavideofromframetoframe.Figure5showsasingleframeof
videowiththemotionvectorsoverlaid.
[Link].
ThenextstepinthedetectionprocessistoseparateoutpixelsthataremovingandthenusetheImageRegionAnalyzerapptoanalyzetheconnected
[Link]
fieldofview(Figure6).
Technical Articles and Newsletters
[Link].
Wenowhaveallthepiecesweneedtobuildapetdetectionandrecognitionsystem(Figure7).Thesystemcan:
Detectthelocationofthepetinnewimagesusingopticalflow
CropthepetfromtheimageandextractfeaturesusingapretrainedCNN
ClassifythefeaturesusingtheSVMclassifierwetrainedtodetermineifthepetisacatoradog
[Link].
[Link]
problemforexample,classifyingtypesofcarsinvideosfortrafficflowanalysis,identifyingtumorsinmassspectrometrydataforcancerresearch,oridentifying
individualsbytheirfacialfeaturesforsecuritysystems.
ArticlefeaturedinMathWorksNews&Notes
Published201693019v00
Products Used
MATLAB
NeuralNetworkToolbox
StatisticsandMachineLearningToolbox
Learn More
ObjectRecognition:DeepLearningandMachineLearningforComputerVision(26:57)
ObjectDetectionExampleCode(download)
DeepLearningin11LinesofMATLABCode(2:38)
View Articles for Related Capabilities
AlgorithmDevelopment
