SoftwareDesign Document

for

UCSB360
Version1.0

Preparedby
GroupName:TeamEpsilon MaxHinson JhonFaghihNassiri LukeBuckland AndyChou JichengHuang 4426771 4111274 4060893 4061123 4088779 maxwellhinson@gmail.com jfaghihnassiri@gmail.com bigduker20@yahoo.com andy168chipz@gmail.com jicheng0903@gmail.com

Instructor: Course: LabSection: TeachingAssistant: Date:

ChandraKrintz CS189A Friday12PM StratosDimopoulos 03/07/13

RevisionHistory
Version PrimaryAuthor(s) Descriptionof Version Initialversionfor SDD DateCompleted 1.0 TeamEpsilon 03/07/13

TableofContents
RevisionHistory TableofContents 1Introduction 1.1Description 1.2SystemOverviewDiagram 2GraphicalUserInterface 2.1FlowChart 2.2LoginPage 2.3CameraView 2.4Shareview 2.5CreateGraffitiView 3Vuforia 4Facebook 5DatabaseInterfaceandtheMidLevelModule 5.1Overview 5.2DatabaseInterface 5.2.1DatabaseManager 5.3UserInformationManagement 5.3.1User 5.3.2FriendManager 5.3.3Friend 5.3.4Loadinguserandfriendinformation 5.4TargetManagement 5.4.1TargetManager 5.4.2Target 5.4.3Augmentation 5.4.4Loadingtargetandaugmentationinformation * 6Database 6.1Overview 6.2ERDiagram

1Introduction
1.1Description
ThisdocumentdescribesthesystemdesignfortheUCSB360application.Topicsthatwillbe coveredrangefromhighlevelsystemarchitectureanddataflowtolowlevelclassdescriptions. Thisdocumentwilldescribeeachmoduleindetail,andwillincludeUMLdiagrams,timing diagrams,andPRDtables. Additionally,preliminarytestcaseswillbeprovidedinaseparatezipfile.Thisfilewillcontaintwo typesoftestcases:manualandJUnit.Detailed,writtenuserinterfacetestswillbeperformed manuallybytheteam.JUnittestcaseswillberunonthemidandlowlevelJavacode.Both typesoftestsshallberunregularlytoensurethatimplementationadherestothedesign.

1.2SystemOverviewDiagram

2GraphicalUserInterface
2.1FlowChart

2.2LoginPage
userwillhaveachoicebetweenlogintoFacebookorusetheapplicationwithoutloginin. Ifuserselectstologinwithfacebookaccount,userwillbedirectedtoaloginpageprovidedby FacebookAPI.Afterauthorizationissuccessful,userwillbedirectedtothecameraviewand startviewing.Bycontrast,ifuserclickedviewwithoutlogin,hewillbedirectedtothecamera viewimmediatebutresultsinnotallowedtoshareviewsuserhasseenwithhis/herfriends.

2.3CameraView
Adropdownmenuwillbeavailabletouserontopofthisscreen,usercanslidedownthe dropbuttonincasehe/sheneedstodosomesettingabouttheapplicationorreadsomehelp tips.Wheneveruserwantstostartviewingaugmentation,hecanjustsingletapthescreen.User willseeascanningbarrunningbackandforthonthescreeninordertotellthesystemis 5

running,andeverythingatargetisrecognizedfromthecameratherewillbegreendotsshowup onthetarget.Renderingwillstartassoonasthetargetisrecognized,andtheaugmentationof thecorrespondingtargetwillshowupwhenfinishedloadingfromthecloud.Byclickingthe sharebuttonlocatedatbottomleftcornerofthescreen,thesystemwilltakeasnapshotofthe currentviewuserisseeinganddirectstotheshareview,andclickingthetagbuttononbottom rightcorner,systemwilltakeasnapshotofonlythetargetfromthecameraviewanddirectuser tothecreategraffitiview.

2.4Shareview
Userwillbeabletoseethesnapshotsystemtookanddecidedtopostonhisfacebook wallbyclickingthesharetofacebookbutton.Thiswilldirectusertothesharepageprovidedby FacebookAPIandposttowallprocesswillbedoneinthatpage. Or,usercansharetheviewwithhisfriendswithinthisapplicationbyclickingtheshareto friendsbutton.

2.5CreateGraffitiView
Thecapturedtargetwillbeshownonthescreen,andpaintingtoolswillbeavailableatthe bottomofthescreen,usercanclickoneachtoolicontousethemforcreatingtheaugmentation. Thereisacolorpickerbuttonontherightsideofthetoolicons,andascrollbarforzoominand zoomoutwillbeavailableontherightsideofthescreen.Onceuserfinisheddrawinghis graffiti/augmentation,hecanclickthedonebuttonanddirectstotheshareviewtoseehowthe finalresultwilllooklikeanddecideifhewantstoshareitwithhisfriendsornot.

2.6GUIFunctions
ActionBar() AbuildinclassofAndroidwhichisusedtoholdallthetoolsforcreatingagraffiti. Containsbuttonslike:Shape,Brush,Pen,Line,color,etc.Eachbuttonwillcontainaspinner whichwilllistallpossibleshapesifuserclicksonshapebutton,orsizeofdifferentbrushes whenuserclicksbrushbutton. Animation() AbuildinclassofAndroidwhichisusedforallerrorwarnings.Dialogboxwillshowupin middleofthescreenwheneverusermadeamistakeactionthattheprogramisnotallowedto perform,andthedialogboxwilldisappear/fadeawayautomaticallyafterasettimeperiod. Texture() Aclassthatcontainsallinformationabouttheaugmentationmetadata.Whenevera targetisrecognizedfromthescreen,itscorrespondingmetadatawillbestoredinthetexture class.Metadatawillbeloadedintoabitmapinordertodrawitonthetarget.

onClick() ThisisabuildinmethodofAndroidwhichwillperformdifferentoperationsdependingon whichbuttonhasbeingclickedonthecurrentview.Differentintentwillbecalledinorderto switchbetweendifferentscreenoncethecorrespondingbuttonsareclicked.

3Vuforia
3.1OpeningthecameraandinitializingVuforia
3.1.1Overview
Byopeningtheviewertheusercan: Viewtargetsthroughthecamera Tracktargetsandviewtheresultingaugmentation OninitializationtheviewerorCloudRecoactivity: Initializestheneededvariablesinthejavacode InitializestheinterfacebetweentheCandjavacode Initializestheanopenglrendererandstartsitsoperation InitializestheneededveriablesintheCcode Initializesthecamera InitializestheQCARlibrary Initializesthetrackingoftargets Initializestherenderingoftrackedtargetsoverthecameraview

3.1.2OnCreate()
OncreationtheCloudRecoactivitybeginstheinitializationofthecameraandaugmentationby callingthefunctiontoupdatetheapplicationstatustotheinitializationmode.

3.1.3initApplication()
Thefirststepininitializationmodeistocheckwhatthescreenorientationisandthensaveitas avariable.TheupdateActivityOrientationfunctioniscalledwhichgetsthecurrentconfigurationof thescreenandthensetsflagstoindicatethecurrentorientationofthescreen.Thefunction storeScreenDimensionsisalsocalledwhichqueriesthescreenwidthandheightandthen storesthatinvariablestoo.

3.1.4InitQCARTask()
TheQCARlibraryiswhatthecodeusestoaccessthevuforiafunctions.Tobegininitializationof theQCARlibraryanewInitQCARTaskclassiscreatedandthenexecuted.Thisclasscallsthe QCAR.init()functionandthenloops,checkingtoseeifitisfinished,untilitis.Ifthelibraryfailsto 9

loadthenanerrorisreturnedandtheapplicationexits.

3.1.5initTracker()
InitTrackeristhefirstnativeCfunctiontobecalledfromthejavacode.TheinitTrackerfunctionin CloudReco.cppcallstheQCAR::TrackerManager::getInstance()functiontogetareferenceto thecurrenttrackerManager.FromtheretheinitTrackerfunctionfromthetrackerManagerclassis calledwhichinitializesanimageTrackeraspartofthetrackerManagerandthenreturnsthe addressoftheimageTracker.Thisisstoredinapointerwhichisthencheckedtomakesureit containsanaddress,indicatingtheimageTrackerwasproperlycreated.Thefunctionreturns1 onsuccessand0onfailure.

3.1.6initAppicationAR()
IfinitTrackerreturnssuccesstheactivitymovesontoinitializingtheaugmentedreality.Todo thisitcallstheinitApplicationARfunctionwhichinturncallsthenativeinitApplicationNative functionwhilepassingitthescreenwidthandheight.

3.1.6.1initApplicationNative()
ThisnativefunctionstartsoffbycallingQCAR::setHinttosetthenumberofsimultaneousimage targets.Itthenusestheenv>GetObjectClass()functiontoreturnahandletothe CloudReco.javaclasswhichisstoredtoallowedaccesstothejavafunctionsandclassesfrom theC.TheQCAR::registerCallback()functionisthenusedtotelltheQCARlibrarythatwhenever atrackingcycleisfinishedandanewARstateisavailabletheupdateCallbackfunctionshould becalled.ThefunctionthenstoresthescreenwidthandheightinaQCAR::Vec2fvector variable.ItinitializestheindividualhandlestotheshowErrorMessage,createProductTextureand getProductTexturefunctionsinthejavacodeusingtheenv>GetMethodID()function.Finallyit initializestheflagsthatindicatethestateoftheprogrambycallingtheinitStateVeriables() function.

3.1.6.2returnfrominitApplicationNative()
OnreturningfrominitApplicationNativetheinitApplicationARfunctioninitializesa QCARSampleGLViewclass.Thiscontainstherenderedaugumentationandinitializespartsof theopenGlglview.ThecorrespondingopenGlrendereristhencreatedandaddedtotheclass.A layoutistheninflatedtooverlaythecameraandaloadingbarisaddedtothevisiblepartofthe layout.

3.1.7InitCloudRecoTask()
10

Thenextstepintheinitializationistobeginthesearchingfortargets.Todothisan InitCloudRecoTaskisexecuted.ThisclassonexecutionrunstheNativeinitCloudReco()function inthebackground.

3.1.7.1initCloudReco()
Thisnativefunctionstartsoffbygettingareferencetothetrackermanagerwith QCAR::TrackerManager::getInstance().Itthenusesthisreferenceandthe trackerManager.getTracker()functiontogettheaddressoftherelevantimageTracker.Itstores thisinapointerandthencallsgetTargetFinderonthispointer.Oncethetargetfinderisstoredin anotherpointeritisusedtocallthestartInit()functionwiththerelevantaccesskeysofthecloud database.Oncefinishedthisischeckedtomakesureitproperlyinitializesthetargetfinderwith thecloudstargetinformation.Theresultcodeisreturned.

3.1.7.2returnfrominitCloudReco()
OnreturnfrominitCloudRecotheInitCloudRecoTaskcheckstomakesureitreturnedcorrectly andsendsoutanerrorwhiletheapplicationexitsifnot.

3.1.8Activatingrendererandaddingcontentview
OnreturningfromtheInitCloudRecoTasktheactivityactivatestherendererinthe aforementionedQCARSampleGLViewclass.ItthenaddstheopenGLsurfaceviewtotheview screenwiththeaddContentViewfunction.

3.1.9startCamera()
Thefinalstepininitializingthecameraandaugmentationviewisstartingthecamera.Thisis donewiththestartCameranativefunction.Thisfunctionbeginsbyusing QCAR::CameraDevice::getInstance().init()toinitializethedevice.Itthencallsthe configureVideoBackground()function.Thisbeginsbygettingareferencetothecameradevice withQCAR::CameraDevice::getInstance().Itusesthistostorethevideomodeofthecamera.It theninitializesabunchofcameravariablesincludingtheheightandwidthofthescreenbased onthefoundorientationofthedevice.OnreturnfromtheconfigureVideoBackgroundfunctionthe defaultvideomodeofthecameradeviceisset.Thecameraisstartedwith QCAR::CameraDevice::getInstance().start().Ifthisfailsthefunctionreturns.Oncethisisdonea referencetothetrackerManagerandthenimageTrackerisobtained.TheimageTracker referenceisusedwiththestart()functionintheclasstostarttheimagetrackerstracking.A referencetothetargetfinderisthenobtainedandthetargetrecognitionstartedwiththe startRecognition()functioninthetargetFinderclass.Thisconcludestheinitializationofthe cameraandaugmentation. 11

12

13

3.2NativeC++coderenderingaugmentationinnew frame
3.2.1Overview
ThenativeC++codeforeachframe: Recognizestargetsthatneedtobetracked Startstrackingthem Refreshesthecamerabackgroundandbackgroundinformationvariables Generatestexturesforeachtarget Rendersthetexturesoverthetargetsintheappropriateorientation

3.2.2updateCallback::QCAR_onUpdate()
ThisfunctioniscalledeverytimeatrackingcycleisfinishedandthereisanewARstate available.FirsttheaddressofthetrackerManagerisrequestedwith QCAR::TrackerManager::getInstance()andstored.Fromtheretheaddressoftheimagetracker isrequestedwithtrackerManager.getTracker(QCAR::Tracker::IMAGE_TRACKER))andstored inapointer.FromherethetargetFinderisrequestedfromtheimageTrackerandthenstoredina pointeraswell.AfunctionsofthetargetFinderclassarecalledtoupdatethesearchresultsfor trackables,getthecountofthesetrackablesandthenaccesstheaddressofeachtrackable referenceindependently.Theaddressofeachtrackableisstoredinapointerandthenusedto createnewtrackerstofolloweachtrackable,andalsoaccessmetadataaboutthetrackable.For eachtrackablethejavafunctioncreateProductTexture()calledandpassedthismetadata.

3.2.2.1createProductTexturejavafunction
ThemetadatacontainstheURLofadatabasewhichconnectstrackingtargetswiththeactual augumentationstoberenderedovertopofthem.Theimageisdownloadedandthenconverted intoabitmapwiththeBitmapFactorylibrary.Thisbitmapisthenconvertedintoabufferofpixels andwhichisthenloadedintoatextureformat.FinallyproductTextureIsCreated()iscalledtoseta flagintheCcodetoindicatethetextureiscreated.

3.2.3updateRenderView()
AfterthetrackershavebeenstartedandthetexturescreatedtheCcoderunsthe updateRenderView()functioninCloudReco.java.Thisfunctionstartsoffbystoringthecurrent 14

screenrotation.Fromthereitstoresthescreendimensionsaswell.Itthencallsthenative functionupdateRendering()andpassesitthescreenheightandwidth.

3.2.3.1updateRendering()
TheupdateRenderingfunctionstoresthescreendimensionsintheCcodeandcalls configureVideoBackground().Thisbeginsbygettingareferencetothecameradevicewith QCAR::CameraDevice::getInstance().Itusesthistostorethevideomodeofthecamera.Itthen initializesabunchofcameravariablesincludingtheheightandwidthofthescreenbasedonthe foundorientationofthedevice.

3.2.3.2ReturnfromupdateRendering()
OnreturnfromupdateRenderingtheupdateRenderViewfunctioncallsthenative setProjectionMatrix()function.Thisfunctiongetsareferencetothecurrentcameracalibration withQCAR::CameraDevice::getInstance().getCameraCalibration().Withthisitgetsareference totheprojectionmatrixandstoresitlikeso:projectionMatrix= QCAR::Tool::getProjectionGL(cameraCalibration,2.0f,2500.0f).Projectionmatrixissimply storedinaglobalveriableforfutureuse.UponreturnfromsetProjectionMatrixupdateRendering exits.

3.2.4renderFrame()
ThenextstepintheCcodeistorunthefunctionrenderFrame().Thisfunctionstartsoffby storingthecurrentstateoftheapplicationandstoringitasthebeggingoftherenderingstate. Thevideobackgroundisthendrawnwith QCAR::Renderer::getInstance().drawVideoBackground().Fromtherethefunction generateProductTextureInOpenGL()iscalled.

3.2.4.1generateProductTextureInOpenGL()
Thisfunctionstartsoffbygettingareferencetothetextureobjectcreatedinjava.Itthenuses Texture::createtoconvertittoanopenGLtexture.ThetextureisthengeneratedinopenGLusing severalcommands.

3.2.4.2returntorenderFrame()
UponreturningtorenderFrameitrunsstate.getNumTrackableResults().Thefunctionthenloops doingthefollowingforeverytrackableresult.Itfirstgetsareferencetoeachtrackableresultwith state.getTrackableResult().Itthengetstheposeofeachresultandconvertsittoa modelViewMatrixwithQCAR::Tool::convertPose2GLMatrix()whichitthenstoresforlateruse. 15

NexttheaddressofthetrackableResultitpassedtorenderAugmentation().

3.2.4.3renderAugumentation()
Thisfunctionstartsoffbyscalingtheposeoftheplanerelativetothetarget.Itthenappliesthe necessary3dtransformationstotheplane.Fromthereitgeneratesvertexsforthenecessary points,enablestheseinopenGLandthenenablesblendinginopenGL.Itthenactivatesthe relevanttexture,bindsitanddrawstheelements.FinallyitcleansupalltheopenGLreferences.

3.2.4.4returntorenderFrame()
UponreturntorenderFramethefunctiondisablesthevariousvertexArraysusedintherendering andthensignalstheendoftherenderingwithQCAR::Renderer::getInstance().end().

16

17

18

19

4Facebook
4.1LoginIntroduction
OneofthemainfeaturesofUCSB360istheabilitytologinwithFacebook.Byintegrating withtheFacebookAndroidSDK3.0,UCSB360allowsuserstoperformbasicFacebook relatedtasks.

4.1.1LoginOverview
ByclickingontheFacebookloginbutton,theusercan: Typeinuseremailandpasswordtologintofacebook proceedtoviewtargetsthroughcameraasshowninpart3 OninitializationintegratingwithFacebookSDK: Storeuserinformation(Name,Birthday,Location,FriendList) InitializeUserinfo InitializeFriendList

4.1.2 onCreate()
Thismethodwillinitializethebuttonusedontheloginpage.

4.1.3 createSession()
Createsafacebooksession.Ifnosessionispresent,themethodwillcreateanewone andpassreturnasessionobject.

4.1.4 onActivityResult()
Thismethodiscalledaftertheuserlogsinfacebook,itwilldirecttheusertothetargetview. Iftheuserisnotloggedin,nothingwillhappen. BycallingthefacebookonActivityResulttooverridetheAndroidonActivityResult,the methodisabletogetthecurrentstateofthefacebooktoken.

20

4.1.5 onRestoreInstanceState
ThismethodiscalledafteronStart(),whentheactivityisbeingreinitialized.Thismethodis adefaultimplementationbytheAndroidSDK.

4.1.6 onSaveInstanceState
Thismethodiscalledbeforeanactivityisbeingkilled,sothatthestatecanberestoredin thenextonCreate()methodcall.ThismethodisadefaultimplementationbytheAndroid SDK.

4.1.7 onClick()
ThisisabuiltinmethodbyAndroid.Thismethodisusedtoimplementthefunctionalityof thetwobuttonsonthemainmenu.EachofthebuttonswilldirecttheusertoadifferentUI page.Specifically,logininwithfacebookwillattempttoconnecttofacebook,itwillcreate anerrormessageiftheuserstopstheloginprocessanytimethroughtheloginprocess.

21

22

5DatabaseInterfaceandtheMidLevel Module
5.1Overview
AnothermajorcomponentintheUCSB360systemisthedatabase.Thedatabasestoresall informationforusers,friends,targets,andaugmentations.Itisimperativethattheapplication movesdatabackandforthbetweentheapplicationandthedatabaseasefficientlyaspossible.A groupofmidlevelinterfaceclasseshasbeencreatedtoachievethisgoal. TheFriend,Target,andAugmentationclassesarenothingmorethansimplestoragecontainers fordatathatisloadedfromthedatabase. EachManagerclassconsistsentirelyofstaticmembersandmethods,whichmitigatestheneed forinstantiation.Thesemanagerclassestypicallycontaincollectionsofinstantiatedclassesthat holdotherdata.Mostimportantly,theyservetomediatethetransferofdatabetweenthe databaseandtherestofthesystem. TheUser,FriendManager,andTargetManagerclassesactasaninterfacebetweentheloaded dataandtherestofthesystem.ThismeansthataccesstotheDatabaseManager,Friend, Target,andAugmentationclassesshouldbeentirelycontainedwithinthismodule. JavaDocdocumentationforthesemodulesareincludedinaseparatezipfile.

23

5.2DatabaseInterface
5.2.1DatabaseManager
TheDatabaseManagerclassmediatesalltraffictoandfromthedatabase.Itcontainsthedriver andconnectioninformationneededtoconnecttotheMySQLdatabase.Itimplementsmany helperfunctionsthatallowtheotherManagerclassestoloadinformationfromthedatabase.

24

5.3UserInformationManagement
5.3.1User
TheUserclassisastaticclassthatholdsinformationabouttheuserthatiscurrentlyloggedin. Whentheuserlogsin,theUserobjectmakesfunctioncallstotheDatabaseManagertoload userinformation.ItthenactivatestheFriendManager.

5.3.2FriendManager
TheFriendManagerclassisastaticclassthatmanagesallfriendsthattheuserhasadded.It containsalistofFriendobjects,whichitpopulatesviafunctioncallstotheDatabaseManager andbyaddingfriendsatruntime.

5.3.3Friend
TheFriendclassisasimplecontainerthatstoresallinformationrelatedtoausersfriend.Itis instantiatedwhenloadingausersfriendslistonstartupandwhenauseraddsanewfriend duringruntime.

5.3.4Loadinguserandfriendinformation

25

5.4TargetManagement
5.4.1TargetManager
TheTargetManagerclassisastaticclassthatactsasaninterfacebetweenloadedtargetsand therestofthesystem.ThisclassinteractswiththeDatabaseManagertoloadtargetinformation fromthedatabase.Additionally,itprovidesfunctionsthatallowothermodulestoretrieve informationforloadedtargets.

5.4.2Target
TheTargetclassholdsinformationpertainingtoatargetthathasbeenrecognizedbytheVuforia module.Thisincludesalistoftheaugmentationsassociatedwiththetarget.

5.4.3Augmentation
TheAugmentationclassholdsinformationpertainingtoasingleaugmentation.Mostimportantly, thisclassholdstheURLfortheaugmentationimagethatisstoredintheAmazonWebServices S3filesystem.TheVuforiamodulewilldownloadthisfileandrenderitontheimagetarget.

5.4.4Loadingtargetandaugmentationinformation

26

6Database
6.1Overview
TheUCSB360applicationshallinteractwithaMySQLdatabasehostedontheAmazonWeb ServicesRelationalDatabaseSystem.Thisdatabasewillstoreinformationaboutusers,targets, andaugmentations.SpecificdetailsaboutthestructureofthedatabaseareshowninSection 6.2.

6.2ERDiagram

27