Scribd Logo
Upload

Welcome to Scribd!

  • V

    Uploaded by

    Meesh Valerio
    5 views13 pages
    v

    Original Title

    v

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Report this Document
    v

    Copyright:

    © All Rights Reserved
    Download as PDF or read online from Scribd
    Flag for inappropriate content
    5 views13 pages

    V

    Uploaded by

    Meesh Valerio
    v

    Copyright:

    © All Rights Reserved
    Download as PDF or read online from Scribd
    Flag for inappropriate content
    of 13
    2182017 Patents na Method for real time video processing for changing proportions of an object in the video Us 20150221118 At ABSTRACT Method involving providing an objoct inthe video that at least partly and at least occasional is resented in frames ofa video, detecting the object nthe ‘ideo, whorin si detection compas detecting fea rofrence points ofthe object racking the detected objet in the video, wherein the tracking compeises ‘retin a mesh thats based onthe detected feature reference pots of he object and aligning the mesh othe object in ach rame; ganraing fst set of rode pons onthe creates mesh based on a request fr changing propadions; (gonerating a second set of node pons based onthe frst st of node pls; and transforming th frames ofthe video in such way thatthe objects proportions ae transformed in accordence with tho second set of he nde pont using the mesh, IMAGES (10) [FP aiei Drown) [ZPaictOrving [Pale Ding DESCRIPTION (CROSS-REFERENCE TO RELATED APPLICATIONS [000%] This appicationclims the bene of U.S. Provisional Appleation No, 61/996,076, fed on Feo, 28, 2014 BACKGROUND OF THE INVENTION [0002 1, Teotiea ea [0003] The asclosed embodiments relate general othe fl of realtime video processing, in particular, to @ system and method of el tine ‘ideo processing fr changing proparions of an cet inthe vide, 2 Description of the Related At Nowadays a variety of devices and programs can provide processing of til images, or example etfocs ik fae thirvig, makswp te, and processing of rea time video using Some titers (or example, web cam ideo), Also some fae racking algorithms and implemartatons for ‘Video streams or vdeo data are kn In parted, some programs can change an object na video stream, for exams, change a person's face by changing proportions oa whale frame or overlaying any extra abject on a person’ face. However, ‘hore reno programs that can implement changes to an object in @ ‘video stream that sem fo be natal and cannot be racogrized wth the naked oye. Further, such programs cannot bo implemented in ral ime by mobile devies, since they ar escurcesntensive and such devices not handle sc operation fr changing an objet in al tine, {0004 {0008 [00071 U.S. Patent Appleton Pubcaton No. USZ007268812, Incorporated ipsuhwor goole.comfpatersiUS20150221118 Publication numba tert US20180221118 - Method for eal ime video processing fo changing proportion of an objec.» Google Patents Downiontor Rw uszotsoz21118 81 [zPatent Orang) Publication type -Apoieation ‘Application number US ta/3t4,312 Publication date ‘ug 2018 ing date shan 28, 2014 Priority date Feb 5, 2004 ‘Also published as 50308525, S More» Inventors elena Shaburovs Original Assignee Elena Shaburove Export citation BIBTEX, EncNate, RefMan Patent Gtatons (16), Referenced by (4, Classifications (4 Legal Evonts (2) Extoral Links: USPTO, USPTO Assignment, Espacenet (Patent Drawing] [)Psieré Orowing) [ZPalert Drawing Pe CLAIMS (21) Whats climes is 4.8 compar implemented metho fr rate video processing for changing proportions of an abject inthe vido, the methed comprising roving an abject in the vide that at last partly and atleast ‘occasionally is presented in tames of the video: detecting tho object n the video, wherein sas detection comprises detecting feature reference points of the object ‘racking the detected objet inte vieo, whorin the tracking comprises ‘rating a mesh that is based onthe datocted featur eerence points of the ‘objet and algrng the mesh to the object in each frame, ‘generating a fist set of nae points onthe created mesh based on a request for changing proportions, ‘generating a second set of node points based onthe st set of node pants ond ‘wansfoming the ames ofthe vdeo such way tha he objets ropations are transformed in accerdance with he second set the node Points using the mesh, 2. The computer implemented mated of clam 1, futher comsrsing: ws 2182017 {0008 tert US20180221118 - Method for eal ime video processing fo changing proportion of an objec.» Google Patents This metned involves changing ofan object in a video stream by overlaying it with new predaternined images. However itis nat possible to process real time video such tha an cbject shown in eal time vdeo canbe mei in ral time naturally ity some afects. case of # human's face such elects can include making a face fattorthioner a8 well a char dstans Thus, now ard improved systems and methods are needed that woud enable real time vio processing fx changing proparons of an cbect inthe video, ‘SUMMARY OF THE INVENTION {00094 (oot) (0012) [oot] (00%) [00t5) (00%6) (0017 ‘The embodiments deserted herein ara rected to systems and ‘methods that substataly obviate cre or more of the above and ober problems assorted wih the converenal technology fo eal ie: ideo processing In accordance with one aspect the embodiments desonded herein, theres provides a computerimplemented method for rel time video processing for changing propotins ofan obec inthe video the ‘method invlvng: providing an abject in he video that at leas paially and at last cccasionaly is presented in amos ofa video: detecting the object nthe video, wherein said detacton comprises detecting feature reterence pont of the object tacking the detected objet in tho vido, wherein tho tacking compris creating a mosh that Hs based onthe detected feature reference polnts ofthe object and llgring the mash tothe oboe in each frame; ganerating a frst set of node pont onthe creates mash based on a request fr changing roparons; generating @ second set of nod pont based onthe est set of noe pont; and transforming the frames ofthe vio in such way that he objects proportions ae tansfermed in accordance with th socond st ofthe nedo points using the mesh. In one or more embasiments, the computernplementod method further comprises ewan a square gid associated wih a background ofthe tect nthe vdeo; and ranstoming he background ofthe abject using the square gato avo he background astortion. In one cr more embciments, the object the vio to be detected i @ human fae, In one or more embeciments, the objects feature reference points are a eas ane of tho penis incating eyebrows vertical poston, eyes ‘vatcal positon, eyes with, eyes height. eye separation distancs nose vertical postin, nose pointing wp, mouth vertical poston, mouth wich, chin with, upper pais jw don ip stotch, brow omer, Hht brow loner, ip comer depressr, and outer tow ase. In one a more embeciments, he method futher compdises: ndeating 2 prasence of an abject rom a Ist of objects in rams ofthe vdeo, herein thelist futher compris rls fer changing prepotions of each ject rom thelist; ane generating a request fr changing proportions of tho objoctwrich presence in frames of tho video is indicated. In one oF mare enteciments, the method futher compdses: dering 2 cotject to be changed in ames of the video and tle fr changing Proportions of the atiect by @ use; and generating a request for changing proporins of the ebect dined by the use. In one or more embeciments, he method futher comprises: dering by ‘2 user frame ara f th vdeo tobe processed, wharcin the frame sara to be processed sets frame aoa ofthe vdeo such that only proportions f these objects or thor parts which re postoned in the frame area to be processod are changed In one or mor enteciments, te method futher compdses: randomly selecting a least one object to be changed in frames ofthe vido ou of the objects names ofthe vdeo and randomly selecting at east one le for enargrg proporons of he solace object out of alist of rules; land generating a request er changing proportions ofthe randomly ipsuhwor goole.comfpatersiUS20150221118 crating a sau the vdeo: and 9 associated witha background ofthe object in ‘transforming the background the aboot using the square gi to «vai the background cistron 83. The computer implemented mathe of clam 1, when tho object in ‘he video tobe dotected ie 8 human fac 4. The compxter implemented method of claim 3, wherin the object's {coture reference points se atleast ane ofthe pos indicating ‘eyebrows vertical position, eyes vertical positon, eyes wat, eyes hat, eye seperation dstance, nose verte! poston, nose porting Up, ‘vou verbal postion, mouth wth, ein wih, upper i alee, hp, ip stots, let brow lowers, ght brow loweer, ip comer pressor, and outer row ais 5. The computer implemented method of lam 1, wherein the method ‘futher compcees inscating a prosonce ofan objet fom a tof objects in frames of the vdeo, wharsin the Ist ures comprises les for changing proportions ofeach object rom the Ist and generating a reqest for chaning proprtins of the ate! wren presence in ames of he vdeo i nested. 6. The comptes implemented method of lam 1, wherein the metho 7 R te (x, y2) OR 18 (x, ¥21(22) [0112] Then interpolating inthe rections made [0009] fF (P)ay2-yy2-y ATED (RA)eyey ty2-y ITO (R2}(2) [0000] This gives the dost estimate of. [0000] F(x. y P(x 2-x)L Cy Dey )(x2—K A) ACY YALE ICR y 1) (Xx AYO LY Rey (xD-R 1) (yy APE (K2 NYHA IY AZATIE (Bey ALLS OC Y2)* (XK 1) Voy VDA RY OLN (2, 72) (24) [013] Red pcs as the veicas ofthe gr are shown in FIG. (0778) one.or more embodiments, to make calculations faster the values of ware beng pre-calculated foal eget ‘vectors pv inthe besnning ofthe program work and eal values arena sing ealuated during algorihm work ‘They 2s beng takon by the nearest neighbor method [0715] In one or more embociments, foreach picl of he ruting point its value fe calculated using the next formula: [0000] 6u=S Cu sx-tra(vjex Cet &sdury-feL (yyy det cov (tafe (vp eux) (1 f(y) youeyO)E Cw eetrs (ve K IT&C wy feo (vey ee o Fray yew KO) (Luteo (v)-yeu-y a) (25) [011] where u's 2 poet onthe resuting image, vs @ pat onthe nal Image, e418 calor of peel u, cys a color of ka To fin all the picle on tne intial image which satity the condtion {00004 ipsuhwor goole.comfpatersiUS20150221118 ars 2182017 own [00004 {00004 {00004 (00001 ong {00004 (0000) (00004 {00004 ong, (0129) low (0124 (02a [o124 (0725) (0726 (own (0728 (0123) (0129) [0134 (0132 (0139 [0124 (0129 (0133 (0137 (0138 (0138) roo) [ows (oa rou] ro1aq (045, (07464 (own (074s) (o%4sy (0150) Part US20180221118 - Method for realtime video processing for changing proportions ofan object rot necessary to look threugh al he pixels. Instead th transfomaton fis bul and foreach pif the nearest pixels ae fount: Maer) en) Wamanrann) (8) Mrorater iy) (29) Wane does) (20) land save two conesponding sums for hem: buflerSumstueedtFyrmwaDI-Fivy-uy) 1) buflerWogrtulee(idvauAKiH dey) (32) ‘Tenth color value each pixel canbe calculate as folowing: (cu= buferSums i [u baerWeight [u) (29) I some resuting points dst have aprotetypo, ther values are called using bilinear Interpolation on neighbors In one or moro embesiments, face racking reslls aro usd to choose sats of contel points p and q, Some) ‘vetoes of Candie are projected tothe plane and 8 points are add: 4 comer pins ad # milo of borders. “This sot of points is taken as p, On tho FIG, 6 the choice of contol pons (markod ron) is shown. none or more entciments, 10 obtain seta Defemation unis to Candide were intaduced. Thay are some paramatess that corespond tothe desired deformations. In ths emBaiment 3 deormation units are added Fatness Nose wish Eyewiats However, n ther enbodiments athe defarmstion units can be chosen to implement the desved face deformation In one or mar enteciments, each of Deformation uit iluencas on some Candle pins piers and thas Its current valu in each memento ime—he bigger value, the bigger iver. For examplo, fo make a man fate, Faness value shoudl be increased and to make tim thinner it shodd be decreased. ‘Thus, each moment of time wo Canad models with egal values of Shape and Action unts are present, but wit ferent values of Deformation uns. The fst Candid corespords tothe real face form and the secon one caresponds othe wanted form. By the second Candide ports’ projection tothe plane set qs obtained. On the FIG, 7 the diference between sets p green pont) and q(corespondng blue pins is shown, Than MLS is used to get ransfomation of pinto a Hor ae values of Deformation unis fluence on the chosen paints inthe deserted embodiment Falness (8) 152 0,050000 0,090000 0.000000 '51 0.100000 0.000000 0.000000 63 0110000 0.000000 0.000000, 29 -0.050000 0.000000 0.000000 2a -0.100000 0.000000 0.000000, 0 -0.110000 0.000000 0.000000 165 0.000000 0.190000 0.000000 -220,000000 0.190000 0.000000 Nowe with (4) 75 0,050000 0,000000 0.000000 175 -0,0500000,0000000,000000, 73 0.030000 0.000000 0.000000 7 -0.020000 0.000000 0.000000, Eye wit (10) 52 0,000000 0.020000 0.000000 652 -0.020000 0.000000 0.000000, 156 0020000 0.090000 0.000000 57 0,000000 ~0.030000 0.000000, 73.0,.000000 0.025000 0.000000 +8 0.000000 0.030000 0.000000, 20 0.020000 0.000000 0.000000 ipsuhwor goole.comfpatersiUS20150221118 Google Patents as 2182017 tert US20180221118 - Method for eal ime video processing fo changing proportion of an objec.» Google Patents [0153] 0.000000 0.028000 0.000000, [0184] Examples of Fatnass and Nose width deformations’ aplying are shown in FIGS. 8{a}-8(). To make fat 4etormaton more natural mouths net steicned wile making peopl ater bul mouth is compressed while making people thinner (0155) Ths, the agora to: [0158] 1 tnd the Cardi poston (Shape and Acton writs) [0157] 2 spply Defamation uns othe second Candide [0158] 2. project both Candaes to cbain set p and q [0159] 4, bud the deformation using MLS in gid vations [0160] 5 calculate defamation in ll pixels using bilnoar interpolation [0167 6. ld resulting pete (0162) none or more embodiments, to make thi eect eal time GPU is used with some optimizations ofits fuetioning. The image is $i wth gular grt ares the transformation alles only nis rods, Then the "nea interpolation ' used to get transformation at each pixel. With increasing of ga size ps (rames per second) Is inerwased but qualty becomes worse. [0163] Thus, changing ofthe obec’ propor in el tne in video stream according to te present nvertion has the folowing stinguistig features In the erga lgothm tne invertors have to compute transformation fr each xa, but ona davice it ans slow. T increase speed he invertors cvdeplare of mage wih regulr gid ara campule tansferatio in fd nodes erly Transfematin in cher pixels is intorpolte. [0164] Furtnor advantages of th descr embodiments ae given bythe fact that the method of el ime vdoo stam processing for changing proportions ofan obec nthe video stream canbe implemented on mobile devices, for ‘examplo such 2s mobile phones, smartphones, abt compsters ol. since the method isnt resource-nansve. xemplary Computer Platform [0165] FIG. 91 a block dagram tat ustrates an embodiment ofa comguter system 600 upon wre various ‘embodiments othe iventve concept deserted herein may be implemented, The system $00 Includes @ computer platform 801, peripheral devices 502 and network resources 803. [0188] The comeuter platform 801 may include a data bus $04 o other communication machanim fer communicating Infomation across and among various pars ofthe computer pltfer S01, anda processor 506 coupled wih bus '504 or pocessing information and peroring other computational and contra tasks. Computer peter 07 also inches a vole storage 606, such asa random access memory (RAM) of ther dynans storage deve, coupled to bt 504 fo string various inlormaton as wall mF nstctions to bo exeested by processor 505, Including the sofware application for lnplementing mulunctioal erection with elements of alist using toueh- sonstve devices described above. The volatile storage 506 also may be used for string temporary variables or ‘ther intemedat information dung excetion ef etaction by processor S05. Compr platform S04 may futher include read aly memery (ROM or EPROM) 807 or tr static storage device couled to bus 504 for storing static infomation an instructions fr processor 606, such as basi ingu-cuoat systom (BIOS), as well 2s various system configuration parameters. A persistent storage device S08, such as a magratic dk, optical és, or Soldat lash momary dvi is provided ae coupled lo bs 804 fo string information and instructions [0167] Computer lattom 801 may be couple va bus 804 loa touch-sensitive splay 809, such asa cathod ray tubo (ERM) plasma display, ora lui crystal espa (LCD), for displaying information toa systom administer or user af the compute lator 501. An put devi 510, including alphanumeric ana ther Key, coupled to as 1504 for communicating iarmaton and command selections to processor 505. Anather typeof user input devices curt conta doves S11, such asa mouse, rackal, or cursar dection keys for communicating direction infomation ard command selections to processcr 808 and fr contling curser maversrt on touc-sensive splay 509. This input device ypicaly nas two degrees of rod in wo axes, aft axis (9, x) anda second aris (69, y). tha allows the device to specty postions in plane Te detect user's gestures, the display 809 may incorporate a touchscreen ntetace confgured to detect users tactile events and send igermaton on the deloctes ‘vents to the processor 805 via the bus 504 [0168] Anextema storage device 512 may be coupled lo the compu: paorm 501 via bus 504 to provide an extra oF removable storage capacty fr the computer patfom S01. nan embodman: ofthe computer system 500, the extemal emovable storage device S12 may be used to facltate exchange of data with athe computer systems. [0169] The invents related to he use of computer system 500 fr implementing the techniques describe herein. In an ‘embodiment, th inventive system may reside on @ machine such as computer platform 801. According to one ‘embodiment of the invention, the teciniques described herein ae performed by compier system 500 in response to processor 808 exocutng ene or more sequencos of ene or mors instuctions cerained inthe volatile memory 506. Such instructions may be rea into volatile memory 508 fm anole: compuler

    You might also like