Scribd Logo
Upload

Welcome to Scribd!

  • Convo Lu

    Uploaded by

    Alemat Tesfay
    5 views11 pages
    convolution

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Report this Document
    convolution

    Copyright:

    © All Rights Reserved
    Download as PDF or read online from Scribd
    Flag for inappropriate content
    5 views11 pages

    Convo Lu

    Uploaded by

    Alemat Tesfay
    convolution

    Copyright:

    © All Rights Reserved
    Download as PDF or read online from Scribd
    Flag for inappropriate content
    of 11
    2o/10/2017 Understanding Convolutions-colah’s blog Understanding Convolutions Posted on July 13, 2014 neural netaorks o/s /tss/ncurcl_nesworks Mel), convolutional neural —_ networks (//ooetstags/corwolsional_newral_networks hil), convolution (/./pots/tags/convoation Atl), math //oaets/tags/math Mend), probability (/./pots/tags/probablty tl) 1m a previous post (./201407-Con-Nets-Modular/}, we bult up an understanding of convolutional neural networks, without refering to aay signin natbemaaies. To go further, however, we need to nudenstand comvoltions we just wanted co understand coswoutional neural networks, i rnight sulle to roughly wnderstand convoutions. But ‘he ain of this series to bring to the unter of convolutional neural networks and explore new options. To do that, wee gong to need to understand convolutons very deeply. ‘Thanklly, with a fow examples, convolution becomes quite a straightforward idea, Lessons from a Dropped Ball Inagine we drop a bal om some Inght onto the ground, where only has one dnson of motion. How Ml ‘hat bal wl go dtanse © you drops and tho drop apn from abo the point a which i landad? let's teak cis dowa, After the fst drop it wil land @ units away fom the stating pint with probably f(a), where fs the probity dsebuto, Now after this fist drop, me pick the ball up and drop it for another heigit above the point where it fist landed. The probability of the ball rolling b units away from the new starting point is g(b), where g may be a different probability We cops fom a diferent height Aistrbation . a F(a) i> I ww fix the rele of the ist drop so we Know the hall went distance @, forthe ball to go a total distance 6, the Aistsnce traveled in the recon drop sao ix wt, where a + b =. So the probability ofthis happening i simply (a) fb)" {Let's ehiak about this with a specific discrete camsple, We want the tora! distance ¢ to be 8. If ce fist tan it rolls, (a= 2, the second time it nautt roll b= 1 in order to tesch our total distance a +B = 3. The probability of this i 4(2) (2). 20/10/2017 Understancing Convolutons -colah's blog ° ' im However, thie int the only way we ould get to a total distance ofS. ‘The ball cou rol 1 units the frst vie, end 2 the second, Or O units the fst time and all 3 the seoond. Ie oould go any @ and B, as long as they add to 8. ‘The probate are f(1) - (2) and (0) -9(8), respectively In order to fd the total Wkelhood of che ball reahing a total distance of ¢, we can't consider ony ome possible way of reaching €Tastend, we consider all the possible ways of partitioning € into two deope a and B and sm over the rola of cack way £(0)-9(3) + F(1)-9(2) + F(2)-9(1) We steady know thatthe probability fr ech came of a+ B= ie simply f(a) -g(8) So, emming over every solution to @ +B =e, we can denote the total Heslitood as ¥ #9) ‘Turne ont, we're doing a convolution! In parties, the convolution of F and 9, evlsated at i define: (Fr 9)(e) = D2 Ha)-9f6) Awe subetitate = @~ a, we ge: (f+ 9)(2) = > fla) 9(e-a) “This isthe standard defition? of convolution, ‘To mala thie» bit more eon it wil and ata intermediate position a with probability f(a). Hit lands at a it has probability g(¢ — a) of lancting se, wn thik about Chie i erm af potions the tll might land. Afr the first dap, 8 positon 20/10/2017 Understancing Convolutons -colah's blog sla) A ON 66 NN) @ rove-0 ‘To get the convolution, we consider all intermediate positions, ila) @ Tiee-a) Visualizing Convolutions ‘There's very nlc trick chat helps ane think about convolutions more easly Fintan oberation, Suppose the probability tht» bail lands a crtn distance = from wine it stata i f(2) ‘Then, alarm the probability that iad sda @ fom where it anda is f(—2). S(-2) e fle) 20/10/2017 Understancing Convolutons -colah's blog 1 we know the bull lands at a position € after the sond drop, what isthe probability thatthe previous postion was -e a | ome g(-(a-0)) <—_—______ So the probability thatthe previous position was ais g(—(a~ ¢)) = g(e— a), Now, cumider the probity cach imemadiace position consbutes tothe ball Sully landing at €. We know the obebilty ofthe st dop puting the Bll ato the iene poston ais f(a) We abo know thatthe probabity of having ben sa ft Innds a eagle — a) e ibe. F(a) _a | are Fla)ale~a) z “ we-a) Summing over the as, we gt the convolution. “The advantage of this approach is that i allows us to visualize the evaluation of & convolition at a value ¢ in a single picture. By sifting the bo:tom half around, we can evaluate the couvolution at other values of ¢. This allows us to undermand the convolution a8 a whole, For example, we ean ae Ub it peal when the distention align 20/10/2017 Understancing Convolutons -colah's blog 1 eee “Ty And shrinks os the inteeseion betwoen the distributions gots salle uh, t ‘By using this tic in an animation, it eally becomes possible to visually understand convolutions. Below, we're able to vinualize the convolition of tw bos functions 20/10/2017 Understancing Convolutons -colah's blog TSmnarcare ae Sa nd en row From Wikipdia Armed with this perspective, a lot of things become mare intuitive Let's consider a nomprobabiltie example, Convolutions ate sometimes used in sidio manipulation, For example, one vaight use function with two spikes ini, but aro everywhere cls, to create an echo. As our doublespiked fimtion slides, one spe hits a polit in tne fist adding that signal to che output sound, and later, another spike follows, suing a acoond, delayed cop Higher Dimensional Convolutions CConvolutions are an extremely general idea. We can also use them ina higher muaber of dimensions Let's consider oar exasnple of fling ball nga. Nove, 8 tf, i position shifts not only im one diaension, but in 9(b) Convolution i the sane aa before (Fea) =X fla) 910) Bxcapt now a, Band € are vectors. Tobe more exit (Fe o)lerer)= YD er,42)-a(bisa) (rin the standard deistion 20/10/2017 Understancing Convolutons -colah's blog (Ff g)(ere2, De Flare) (er — a1, ¢2 — a2) Jhustlke one-dimensional convolutons, we eax think of « two-dimensional convolution as ling one function on top of another, multiplying and adding. (One common appllzation of this is nage procuaing. We can think of images as two-dimensional functions. Mawy ‘important image transformations are convolution where you convolve the image function with a vary small, oeat TRA rom the River Tel documentation (bt itllab Riera atrial) ‘The kernel slides to every position ofthe image and computes a new pixel asa weighted stun ofthe pies it doats owe, For cxample, by averaging a 3x3 box of plas, we can blur an sage. Todo tis, on kernel takes the value 1/9 on cach pel inthe box, lelsisIsle| lels]s[s[e| Derived fom the Grp doctmntaton (tps dock gmp orgy plug events) ‘We can alto detect edges by taking the values —1 and 1 on two adscant pines, and zero everywhere ele, Th subtract two adjacent pisels, When side by side pixels are similar, this is gives us approximately zero. On exges, however, edjacent pixels ae wey different in the direction perpendicular to the ode, 20/10/2017 Understansing Convolutions -colahs blog Derived fom the Gimp documentation tp doce gop ong plug ovat Te snp doctaneataton he maty other exausples (tp /doesgenpong/en/pu: toon tel). Convolutional Neural Networks 50, how does convolution rete Lo convolutional neural networks? Consider a 1

    You might also like