2 2 a e IMEURAL NETWORKS € DEEP LEARN DEEP LEARNING SUPERVISED LEARNING. AUDIO CONVOLUTIONAL NN NUT HIDDEN Me EMER . detec MM Ca DAL ne 8 iM oro — O20 Pee Sac Dy WeattH — ex Hi Gurrur: 4 _ [NN Tore E==4 J STANDARD: SRUCURED "THe OWICK BRAWN Fok” WIL LUGCON Ad(o/) | _NN ome) CONV. NIN (CAN) 60> TEXT TRANSCRIPT | RECURRENT NN ee CHINESE (enn) WHY NOW? uc am POS #F OER CARS] CUSTOM / crt aad ine kr a =| 3 mew HAS BEEN MOVING & qusscv. —$R0M SI6MOID TO NETWORK RELA FOR FASER =o a OE GRADENT DESCENT IDEA a! oN eee g Evtehln ODE abs roroxc OO a J . FASER COMPHTATION recent AN {S IMPORTANT Th SPEEDUP HE IRERATIWE PROCESS, CestenendeeHMR CASSIRCATON LOGISTIC. REGRESSION [=] as cnr AS A NEURAL NET xX 7 FINDING RE MINIMUM yolrh GRADIENT DESCENT eeatSiow aes ae “3 Sue t SRO DeevarveS) . = + a / Cees - o REPEAT UNTIL pee paceeediad par PUTTING tr ALL TDGETHER THETASK IS TO LEARNIO € b Bir HOW? om x “ps § NEHRAL z Goer =] toe Rae? Ore sun 2 2 2 2 a A. OPTIMIZE 00 GDOD THE GUESS IS BY MINIMIZING IE DF BETWEEN GUESS (D com Mey AND TRUTH a alee) Ge A 6 sueMo1D (2) Wes dC cay 4 BUS cnc aTe 4 Cost= Jtw, Bel 2eGry aterm Faacin” CUDENT De Cosr= L508 THE ENTE AMET FEAT UNTIL ae CO pestenendez“ANDREW NG * WE 2, z A e 3 g 2 LASER NEUPALNET HIDDEN ite LRYER @ one «SOB SO-4 a “plan unit? ACTIVATION FUNCTIONS SIGMO[D TAN NARS GASSER, NORWAZED “ONY USED FOR >GRANENT OUTPUT LAER, DESUENT IS ‘SUPE |S SMALL UAREEISICAL VAL jbexghoxgs b VATION FUNCTION ID OUST / INITWLIZING ine Aer amy we INTTOD | REN TE [DS UNDEF Ais nL GE ALTE Hls LIN .REGR. aes ne DBE THE SANE AND ARN USED LN FER T RARELY EXACTLY Tou FEAURES | USED INFRA Styl: RANDDM INIT WALL OUD NEURAL NETS WHY ACTIVATION 4UNCTIONS 7 RX. on NO ACNVATION — A= Z TL pe, coe jon FSB pus IN al? a) a Th ee ee ered NAN pix 4b) Func HEME BUT ASO NANT THEM 8 SMALL SD Pane 0.01% CO pestenendez, FBENE TINTTINS peep WALL TEE NET GN CONBUTE NEURAL NETS THAT SHALOWW NETS NEED EXP senscecenenronmneman <2 NORELINTS TOM VERY DATA HUNG Oe Nee re 7 | ® 8 . 5 Oy Power 2S | aes / x lofoPmo | inc g w» @ COMPUTE ON GPUs tind S a Lois OF HUPERELRANS WY h LEARNING PATE $F HOBEN UNITS HENERATIONS — CARICE OF ACTIVATION LO WAL FE HDIEN LABERS MOMENTUM Alp wave —THONENES — WORDS — GENTENGS | NIN-BATOH SE FePAIRES REGWARTEATION aq GAT QO Pestenen deeSETING Ur YOUR ML ABP CIASSIC ML DEEP EEARNING [to-DDCO SAMPLES AM SAMPLES TRAN [bev ]TET] [TRIN Pi} G7 wR aan TAT, ULF SHE Pace, extend [EE oe ISTRIBUTION / vores: [ate Te Dev é Tept GHOuLb COVE FROM SAME HISTRIBUTION [DEA (ae EXPCRIME 0 -— Ce Ee ‘HIGH BIBS Minerer® STRAT. REE ERFOR 1% Bh [5% 05% We |oA 207 4h Wrunle BIA te Hess ; * diese VARIANCE 7, ERROR HN THE ML RECIPE HIGH |u. BIGGER NETWORK shy | "Ri roms HIGH |v, MORE DATA Crain) VARIANCE Real cesnies ieNRIZATION Rare ION PREVENTING OVERFIITING — DROPOUT ee DATERAUGMENTATION GENERATE NEW PIGS FROM EXSTIN- i Z REGULARIZATION (ost: JW), a Gy+A lhl; me \ 0 or lsh Lf. RESIUARZATIONI “ef YO feel ss COM A= KEEGY *A Nol, rrReaet RATON € HM {ONE NDNES ARE RANDOM aoe OROPRED 5 ro) ee SIMPLER NETWORKS 705070. «of 0 Ca RAL “ COURSERA Ee “os O-O= So PROBLEM: AFFECTS BOTH AON0 Oo WE GAT SWRER NE Mepse vatunc € LESS CHANLE TO RELY ON SINGLE FEATURES ene IMPROVING DEEP niIMPROVING DEEP NEURAL NETS: COURSERA OPTIMIZING TRAINING NORMAUZING INPUTS ya 2 oe i SUPE CENTER Ste SK AROUND 6,6 EEN SAME Cx -lot i aa USE SRME AvG/VAR TO NORMALIZE DeV/TEST WHY do Wwe Do THIS? Unsneganszed NoRIAt ZED 2 |© IF WE NORMALE WE CAN USE A weet LPREER. LEARNING RATE Ok oi with RA EON GRADIENTS: Ex: DEEP NWO (Lutes) PUL WW + b IF W=/B5 2] 05 Valine OR WB g]els ts Berane IN BOT Cast GRADIENT TAKES A VERS LONG TIME PARTIBL SOLUMON: CHOOSE (NMTAL \AWIES CAREFULLY GRADIENT CHECKING UF Yom Casr OES NOT DECREASE ON TAH TER YOU MAD HAVE A BRERPROP BUG. GRADIENT CHECKINE APRROXINARS THE: GRADIENTS So You CAN VERIFY CALC. [NOTE] onus we WHEN DEBUGCING OWNLE [TF SL800OPTIMIZATION ALGORITHMS MMINI-BATOL GRAD. DESENT WT SPUT QOUR DANE INTO MINI- BATCHES 0 GRAD DAWENT AFTER €AGH BATCH “TH WAY HOU CRA PROGRESS PIER QUST A SHORT WHILE fi a Fe CHOWING THE. MINIBATCH 62E = Sizeem > BATU GRAD DEX. S1ze=4, > STtcHAsTIC oAADDRL FF Wienke BDAASTC ae Los Amt iy | ASEM VEOIREETN TSHORTITER.. WOES Ver wt) Bn anve coo oar IF go HAE SN OSQUATION f so WE GET TO THE ORL TASTER IN} SMbOTH OUT THE CURVE BY TAKING AN ExPonleNTIAL WEIGHFED AVERAGE OF THE DERWATIVES (le: UT HHS wots) RNGProp ~ ROOT MEAN SOUARED > q * Naud HDT UO WONG Sau= bSaw + G-$) din? oe ae vinx. beb-x te, Wwewage b-b-a SLoneR UERENING ADAN OPTIMIZATION COMBO OF GD W MOMENTUM © RusProp [EARNING RATE DEAS (DEA: SE A LARGE 0¢ IN THE EGINNING-FHEN DECREASE AS WE GET CUBR GOAL UPTON: = SER EAPONENTIALS OX=9.05°""0L, roe Ke OFTinL 3+ X= = oe OPMON4: Oe ferme eros DISRETE SARE OFMN 6: MANUAL E Poci= Rass THROUGH THe enSERA, ING DEEP NEURAL NETS HUPERFARAM TUNING APPRORRIE WHOL HFERRARAMS ARE in OST IMPORTANT ” HAHDDEN UNITS | thet & LEARN capainae | ® Palit IN fb MoMENTAM THRN=0%) OX LEARNING RATE # LASERS LEARNING RATE DECAY 0.0801 0001 0.01 0:1 4 y prod Gr0959 eel6® (AbAW) freee E4o) TASTING VALLES P>EXP EIGET AVE LASSIC )S, 0.39 0299 o HPA - C be-3-rand F370 al PROBIEM): ONE DERATON = C-10") Go. - BUT4 DFE REBALUATE YOUR HUP--PARANS CRD SRC! NOTH nya) © CREO pss PANDA ys CAVIAR ws itt We N oF vas © BABYSTTONE SPAWN LTS OF MODEL CTUNE ODS DEF HP MiRC. BXIRAS BAIL NORMALIZATION NORNALEE LAYER OUTPUT SPEEDS UP TRAINING MAKES WEIGHTS DEEPER IN NiO MORE DBI STCGURIATE) —QUGHT REGUIARIZING ERECT MULTICLASS CLASSIFIC a) Kal calc GT #0 kk OTHER s CwRCASSES = SOEMAX __ “ACTIVATION t=ee™ ca Wn st - 1 p(erHee x) + pCcaT >, +P (FISH rpcec x) 5 OUME7. EY 1498) el=| a4 “J fg Tas aad ft = .0K2| es 70 (EER ah NOY +, [09009 a estateSTRUCTURING ML PROGECTS * COURERA YOUR ML PR RING paECS SEITING YOUR GOAL 7 GOAL SHOULD BE A SINE jPRELGON, RAL [ae3] Al eal KOKAR BREST? jPAETIOOn, ARAL) 4 A | 57 See ht AMS B | 9ex[g57.[917. BEST Ree aoa PTIMIEING VS S ICING METRICS MAXIMIE ACC GIVEN TINE< [Dons ACOURACY= pore RUMTISECNG TING WOUR, wane SETS (yee ORTON UkUs UR UPD el SAM INDIA GIN er lel AUST. IF DEVe TEST ARE DIFF © IDE OPTIMIZE FOR DEV Woe OIL. MISS THE TESFTARET HUMAN LEVEL PERF see A o Adie Cusei) ARNE HUMAN > TRAIN BIGGER NETO. [anne Cran unr /BE8 % C88) TRAIN. ~ GMNE NARGH -HOTERDIRAD TA TRAIN) ] ARH» ROT ARPSTEN, DEV NN ARCHITECURESTRUCTURING ML PROGECTS * COURERA ANALYSIS YOU HAVE ID 7. ERRORS, SOME ARE DOGS MIS GASSIFTED AS CATS: SHOULD You TRAIN ON MORE DD’ PICS? 4. PICK 100 MIS:LABLED -2 COUNT ERROR REASONS _ CAG pi OF ALL ERRORS FOOUSING ON DI6S “THe BEST WE CAN sHbPE FOR 16 O57 ERROR, YOU FIND SDME INGORE. CABLED DATA. IN FRE DEV SET. SHOULD YOU FIX? [ox | DL _ALBORTHMS ARE To RANDOM ER! NOT To S9STEMA (EAA. AIRE CATS INDRR LABLED AS (MICE) Abb EXTRA COL. IN ERROR ANALYSIS AND USE SAME CRIFRE (ARR) Fey woe tru re TEST AS ry NEID PRE - [st SUSTEM QUICK RATE EX: SPEECH RECOGNITION, WOAAT SHOULD YoU FOCUS ON? NOSE: AMCENTS, FAR FROM MIKE @. START QUICKLY DEV/TEST METRICS Q. GET TRAIN-SET TRAIN @. BIAS/VARIANE ANAL ERROR ANALYSIS PRIORITIZE NEXT STEP 2 TESTANSSTRUCTURING ML PROGECTS * COURERA TRAIN ys DEV/TEST BIAS < VARIAN MISMATCH is yismarcied TRAIN/DEV AVMUBEDATA | suman ~or DOD K PROGAT NOS FROM TRAIN 47. ANTERNET , lok puuney carr reay | DEVERR 407. i 1 To Fe Moe ‘ovo o Wa SFLT-STRAN PDBYEST? NOT CENERE IFN OPTONA: GHUFLE ALL ORIS Jey DANA Secon) | se FATE ATF eT “PROBLEM: DEV/TEST 16 Now A as TN oN WOSTLS WEB Mb (beesosenwa)) [Tran PPE] SOUATION: [ET DEW/TEST COME FROM APP THEN SHUTRE 5k b _OF ARP PICS ia WEB FOR TRAN! 10% 206k EE & WEEFAP ‘AP APE liaeaie ae drt ADDRESSING DATA MISMATCH EX. CAR GPS =< TRAINING DATE IS {0.000 SPEECH DATA OF 6 oo OUT MANUAL ERROR, ANALYSIS UNDERSTAND THE DIFFERENGE (EX NOSE STREET NUMBERS) ew MAKE TRAIN MORE SIMIAR TODA) & GAHERMORE DEV-LIE TRAIN: DATA AUDIO 4m © Tcl dee BE CAREFUL: IF You N ONLY HAVE 4 HR OF ¢ APPS IT TO IOK HR SPEECH YOU MAY OVEREIT TD THE CAR NOISE TestaendezSTRUCTURING ML PRODECTS * COURERA EXTENDED LEARNING LEARNING PROBLEM: YoU WANT TO CUPSSIFYGME MEDICAL IMb. you HAVE AN NN THAT CUMSSIFIES CATS 6] (O] (| Ol XO IOH a 6) Sl el elier ec? INN To CIASSIPS CATS /poss/caRs fn Al: OLOGY You ONLY HAVE A FEL [MAGES SOLUTION : |NITUe. WE[CHTS FROM CAT NN eis RETRAIN LAST LAYERS) ON RADICLOSS You HAVE LOTS OF RADICLCES I SQUTION : INIT JOITH WEIGHTS TRIM CHT N RETRAIN AUL LAMERS |S MickasbeT- M VISION. b TRAINING ON MULT. TASKS AT ONCE Pe ‘bere (fest rome. | TTRARIC UcHTLO- UNUKE SOFMAX = MANY THINES CAN BE TRUE =n) IN ‘UTP OPTTONS We GAH, OUST TRAINED 4. NNS INSRAD Bir... MT tenes WAKES SENS WHEN AB He Lepr 2204 bon tev FOR THE DIFF TASES 18. QUITE SIMU — 2 THE AMOUNTS Eb. AK CARS, TE STOPSIENS) 1E SUM OF THE DATA AULOLAS ‘you TD TRA\N A: BiG ENOUGH INN 10 DO WELL DN ALL TASKS COST:3GnB) = WW REAURD TANGER, UARNING 1S Use MORE OFF -END-AD-END LEARNING TRON X‘RAY OF CHILDS HAND TELL ME THE AGE OF THE CHILD Ref D TYPICAL SUN: [. LOCATE BONES TD FIND veNeTS USING ML 2. TRAIN MODEL TO PREDICT ABE BASED ON BONELENGH END-TO-END Hyams cH — Pens tS THe DATA SPEAK, vee I Hl ROUTES WE entree BRSIENING OF oa E vec EXUDES paren ns USEFUL.CONVOLUTIONAL NEURAL NETS -@OURSERK CONVOLUTION Co -1-8-22-5 A CONVOLUTION FUNDAMENTALS Tee 4 2) 2 $5) 3 COMPITER VISION alee 4/2] 1 6/ale Coenen Seeenon Wee 2l4i5|213 19 8 a [ INPUT bb MASE a ce RP RSS pile PROBLEM : IMAGES CN B 2 aR [OCD 1000x3 (2a) 3M aie t WITH 1000-HIMEN UNITS WE IMAGE LE ce | oo 7 SET ure SOWMON : ASE CONVDLLITIONS pean IVS UKE SCANNING ne a Het SHolnG OUTLINES G . TE ae COU HARD-CODE FILTERS GUSTLIKE WE CAN HARDCODE ABURISTIC RULES... BUT:--- A MUCH BETTER ye BM | WH ISTO TREAT THE FILER AS PARAMS SAME TOE LEARNED w]Wa]Ws N IN THE IMb Ns |Ws [We Wa 5 fs FTESPADDING STRID PROBIEM: IMAGES SHRINE WHAT PACE YOU SCAN Wart WUMON: PAD Uo. A BORDER OF @s BEFORE CDNVOLVING CONVOLUTIONAL NEURAL NETS -@OURSERK “STINE, peeisarg s a =A 3} if [a\= [+11 9/4 STS) Near any | Ye 3 T 2 (le)e9j0) — a] | i) al lot ef ]=|9)09) >2] [5] o[sie|=18[ 9/5 PADDING OPTIONS (avo muca To PAX) © Sie fo 0 a Oe ae BG) 3a cur * whee aad Satck 'VALID > P=O bhone| ts 4uous uso DERE FEATURE "SaNE!=> Pafe wader | IN CUR IMAGES FoR GAME We [Rite Au convo uTens ERS CANN BE APPLIED TD 41 ac WELL UKE EKG SIGNALS » AND 3P LUKE CRSANS ines ARE WE VANE TD IND ALL Be TO cp MAUSE ORE BLS MULTPLE FILTERS DETECNING MULTIRE FEATURES AT A TIME S52) rice OkbxS BBS, axa ONE CONV.NET LAWYER +b7 | \ a *by INPUT a UBF g@y~- am . (NOIE] Ut musN'T MATER Ho” BIG THE INPUT IS~THE LEARNABLE PARAMS [) 2 b ID ON THE # OF FILTERS AND THER StetS. 12.3.7 <7 Ob PARAMS ae 2° anbeaeNlCONVOLUTIONAL NEURAL NETS -@OURSERK geo = A ‘7 (| a as | a . FAT tomer Tree Bee aap Ow Te 1080 % lov OF THE Work Is “HIGURING OUT-HOPERPARAMS = HFILTERG STRDE PADDING ETC TYPICAUY SIZE TREND Lown! HPFILIERS 7TREND UP CONVOLUTION POOLING FULLY CONNECTED TYPICAL CONUATET LAYERS ee FIND NAX VAL N SECTION i — ry DETECTED FEAT. MORE ROBUST }-—-APERPARAMS, 2 REDUCES ZiZE OF REPRES- H SPEEDS UP COMPUTATION # MAKES SOME OF THE CONW NET EXAMPLE — DeTECrINe HINDORITTEN DtTs BASED ON LeNet~5 cna Pot Conv. te te | [eS 2 e se pp We CET we LAYER 2 FO ag Poole = 10) fl foCONVOLUTIONAL NEURAL NETS -@OURSERK CLASSIC \IGG-llo CONV. NETS a RU LIRERS HEME SRE eau eas iD PLN LAER 292. s=2 Le! ; ‘ st ie Be DOCUMENT GASIFICATION 2G De PRRAMEFERS easton 2 on Boo} S| Lt 7 ORC os wee Tee Wie 22 ete Conytsel i oo 3: REN: un AY or ats eae a : “7188 M PARAMETERS, Sareea: AO on are —VERY DEEP OID. USED AVG POOLING INT: OF —ERSY ARGHTECTURE See ee ke Ne vOeu CMPON HANES OIE 64)e¢ 5H 52. IP USED SIEMOID/TANH [NC OF RELA Alex Net IMAGE OARSIFCATION x 0M PARAMETERS Kr ‘ioniimap oe ll 2 soos FEE Tae ADE sine Za = SIMILAR To LeNe BUT MUCH Bl66ER, - USES RELU THE NN THAT GIT RESEARUIERS INTERESTED 1p a TH vision AeA RSACONVOLUTIONAL NEURAL NETS -@OURSERK NELWORS.IN NEwOR INCEPTION Gt CONVOLUTION NETWORKS FETINORKC ]5/3[2 PLIOTS|4] | wicrean DF cHOsING A. txh2x8i56 NETO AIOE] ¢ a)-(sl2ieiol | NOP a Ot Give 5¢/ 2/4 116) 4/8 ResNets OVS ol! 0| 6/2] 2 ) Faces arte IrseeMs re races v Seas PROBLEMS 0 ING ee Einibng COBENTS AUAL RES 2 BES ee | [SoumON) RESIDUAL NETS A. NETIORK IN A NETWORK saz MAN pl Gee. 2se : is] grt =| PROBLEM : VERA EXPENSIVE TO COMPUTE a Leen aes 4+ g= ‘SOLUTION: SHRINK THE # CHANNELS Uo H jo aa ow ‘BEFORE APHYING ALL THE z eb = Ab uvtey cguten—val™ 8 | rors Xba a ig at Je | LEARNS OMMEX, NON-LINEAR RELATOOHES s $0 ate ane Deaf) || ABOUT A SUCE OF 4 VOLUME jz rr 2 Pe # CANNES i x ae Bt EEL To PULD AN INCEPTION NEWORK ] YoU MAINLY SACK.A BUNCH OF ae it NN fae ea no-) | | INCEPTION IMOULES es e| Rare “= a eg S ee THE MOVIE —— = kage ae =oe ere ACN > SHE PR L A AS BE ADVICE SIL WANE DTRAIN 4 GASIFIER SPEECH REDS. FOR YOUR CATS BUT DON'T HAVE ENOUGE ATURE [MAGE RECENIToN ese NET BURCH, } Sere SOME OF THE PERS ARE ee y HARD TO IMTIENENT FROM Oh Matar Pee FREEZE THE PARAMS, AND OUST - REAKE 07 SWORE lay LAER VOTH ‘OR DONG DINE RORLET To CONIRIBUTE eae ARKJCOUPETTONS -eemania | rENSEMBUNG Ale GATS FM HCN DATA AUGMENTATION + MST" ATTESTTIME AN WS FAM MULTIPLE THE [MABE WE AuMtgT ALWAYS NEED MORE pi DANA Tp TRAIN ON ; IN PRACTICE THEY ARE Wot USED IN PRODUCTION BECRUSE THEY ARE Covture E vet EXPENSIVE CONVOLUTIONAL NEURAL NETS -@OURSERKCONVOLUTIONAL NEURAL NETS -@OURSERK ALGORITHMS Ee ees a daar OBOECT [oCALIZATION dose (aly (a CAR 7 2 Unwed Like THS LANDMARK DETEMION © F (25 To DETECT LANDMARKS, IN THE FACE CORNER ys gs OF MOUTH ET) LABEL, THE X,Y COORDS OF THE “yg oe Lan bMARK New fis ay [Tosser "0 ‘ena ewe) ams A. BPE TaTeS CUPFED IM Baas ion LINDO OVER THE: IMG. & CLASPS THIS INDOW CRRCYO) AGRINST YOUR reuetesVe) Se SINE ADT ce ie A Lor or CAN 10 THS Hii GERPER 2 cOncLTens mee oo, 55 wes PAS ARE NO) WE OUST PASS. oso TAROWGH ONCE xo _ tines CALC AL BY THERE TMe YOO You Only Lok Once 42 SPLIT IM6 INTO X(3) GRID CEUS ft (QbroR EACH CELL SAY Te Ir CONTHINS CARS, Hae Totatie AONBORD Xe = 3y3KE How Doon KNIbKO +00 EoD wis? ye ga ISTE > Sretor SS WOUs sree 7) INTERSECTION GVER’ UNION GENERALS IF OU 0.5" IT IS REGARDED AG IRREC WHAT IF MULTIPLE SQUARES CLAIM THE SAME CAR?. NON: MAW SUPRESSION IF Tab pou Nel A HIGH as GERD “INCIOR SOKES ANIGHOR BOKES LET YOu ENCODE ine OBOECTS IN THE SAME (C ey ‘SAME ASCONVOLUTIONAL NEURAL NETS -@OURSERK RECOGNITION Yanaron Revue R\(8 A | ae ISTRIS? ML WUT OF K PERDNS) Beir eke Ne, oe ONE-SHor LEARNING HED Dp GE ABLE TD RENE ‘VEN THOUGH WOU CALS fae ene ante I LI DS. you NIT TRAIN ACN WITH ‘SOFTMAX (EACH PERSEND BEOAUSE ‘Ypu DONIT HAVE ENOUGH SAMPLES) IFA NEW FERN UOINS 7D" NEED TD RETRAIN THE NEIWORK LUTION) LEARN A SIMILBRITS INCION orton i BUT HUODO YOU LEARN THIS? SIAMESE WETHORK «= Derrtae Ret [1 sa) See saat er co “Fee ae — PERSON - A(x) 9SMALL a EDX any RIFEREN, PEORE +d Gt po) PLARSE ) Woe cant ACCOMPLISH TARP TE rE LgHioct dx) =1FO9-£69] TRIPLET LOSS FaceNet Pere Pere 20HN intl Tr PRECOMPHTE ENDDINES FAR PAL IN UUP. De S9 YOU Pet Ratt ‘ANCHOR PORTINECE ARORA) NEGATIVE CH) (Gee WANT nga 4Ol< Negfoan® 2 2. dag) (AN) so WE UANTA 68D MAREN Sb it (A,P)-d0y iodeamnraaeadee (s-dAN) +o*s0 . To TRAIN ON? ee we wea Me ect) =f A/P ARE VERY SIMIIAR, © AYN! ARE VERS irreeenr Wee WES ES in we “TRAINING |S. VERY EASY Sos ste eer ‘ SELECT A/N “THAT ARE PRETY SiMIlige TO TRAIN A 06D NET wet ASSCONVOLUTIONAL NEURAL NETS -@OURSERK URAL STYLE TRANSFER cent set) cnc Slit: (ae Re = peer caer acme ey WOE CAN VISUALIZE HATA NEIOCRE iS ti AT Los IMAGES Chis ) LEARNS ACTWARED EACH UNIT) MOST layed | Laure 3 swmle reauees | NoRe comme ee ines edons | remwers Bin | Ceazvnesom | ONE ir ber Bute yon Does THIS HEL? US cee AN IMAGE IN THE SWLE oF ANOTHER? DEA: Ai GENERHEA RANDOM 16 OFT Mee THe OS Feri JOH AT GPL GO guuikane Hoy gmat cee Se6 @ vepnte EACH PIXEL CONTENT CAST TaN UGE A PRE-TRAINED CONVNET xe [66) — SELECT A HIDDEN LANIER SoMeLAHERE JN THe MULE LnTE R= (OPIS LARGER PERTIES = ter a gg pe tae AcrATIONS ate ge g™® age Simiunn THES HAE STR CONTENT eR foam EE How) DAVOE TED. IF Ted ARE SL Tenn "1 IF CAPTURING THE STYLE UING THE STULE IMG AND THE ACTIVANDNS IN A LASER » Look THEDUGH THE ACTUATIONS IN) THe FPERENT CHARIIES 1 GEE HOW CORRELATED THEY ABE WOREN WE SEE OWE MSARLLD SEE IT PATTERNS LIKE THIS WOH Pats ee SE? ae STYLE MATRIX CREATE A MATRIX OF Ho&) CORRELATED “THE ACTVATIONS ARE, FOR EAG PB Gay) © GUNNELPAIE (KK ETE STE He e Seeeared C28 Oia Ae Pe SUE (ost FUNATON JEox[6- 6k ‘Foetus, "NORMQEQUENCE MODELS - COURSERA. CREATE vOCABULAESS (EE PX URAL NE oRDS IN EUR, TEXTS “OR DoUCNLORAD Ext Soe a \i ee ae SEIUENLE PROBLEMS | | aaron] = avid fr HARA PE W our PURPOSE ri ame] seer WM loner FE@ENITION ae |, Seren a reg Bigs | #4 aA [See WE camo use A SmADARD. NETWDRS ) Neu c curPurs CaN wave MRE - Ba Abccccrere| AGeceeTeTe LENGTHS 18 DIFF ED Bethan | Aeonene | anne || WE nt sce PORTURS LEARNED ee] Batkee WORE ACEO. DIERERENT POSITIONS oe a EID | ronnie | eeeacnurs — NEURALNET (a0) | gr Leet gg ly Soe tor (WaME EN "egg g aha ai recounen || ot Lerleely ly ly NANUAD-NANY. Ti #7 TRANSLATION JOTCTore og ge NANEENTETRROBUMION | ety a ieee PREVIOUS RESULTS ARE PASSED IN AS ie r = eeMoINe Ted | INPATS SoWE GET GNTEXT. ep i * i _ Ne co aS?» a (W4CA°IXD+ be) ate . BG PAN ENTRAINED GE? = 9: (Wy A ba) Some a a CS Tc SE Deed 4 0 © ® @ ree a LASS WE OPTI ISTHE Suma OF EXAMPLE TF A Prope WHERE — | 9g .y)-Feom 4-7 @resteventee EVERY XS? HASAN oureut yoMORE ON RNNs VANISHING GRADIENTS ANGUWAGE Moni Teor Ae pr bue vne Pa Ae BI w lu FELLING eS an Be ee THE Of Gypeint ees SINCE LONG SERGE => DEEP RNN HOW Do YOU Knby) IF SOMEDHE SH ‘eae AB Bae a TRE PURPORE TFA Lane. MODEL LS TD CALOAUATE THE PROBABILINES. SS EA.CATS AVERAGE, B HOMRS OF LP ADRS | WOE GET THE VANIGHING GRANENTS PROB. WE ena Stray) [pecoot. | HAVE JN STANDARD NNS ~|£ THE GRADIENTS $ ie : i FOR OMORS HAVE LITLE OF No EFFECT tb ON Vos WERE. 2 Tee, x (NTE) SOMETIMES You SEE EXPLDING GRAD ~ “CAS BVERFIOW NAN) BUT THIS [5 CASILY FIXED Ss) WI GRADIENT CUPPING . pavenons meee Rouen ne res. | SPTEDIREGURRENTHUNIF cou Fee este es, OME HELRS REQALL IF CAT WAS SING. a OR PLURAL SAMPLING SeNrENces iy Alivaes rex eos, | (in) Ate” RAIN : | Q 8 Fano SETA ved (eA) | Ree seches = (rss GEE He NEXT ESTAR? UpdATe cD & oR NOT = AND SAMPLE A NeW WORD Ty fepen UNL x vooras 08 YOU yay! you Ape Nolo U2 REALHED yyy 10 eek: YoUg-OWONI.K-ROOLNE cance Lowug SHORT-TERM MEMORS THE LSM IS A VARIFTION ON THE same THOME 4S GRU BUT WIRY AN ADDITIONAL Te FORGET GATE HE SND, TEMDY BEARS ARE ON SALE HE Smid TEDDY ROOSEVET OAS A GREAT PRESIDENT! PROBLEM: VOITHOLUT LOOKING FOROARD WE GRA ERY IF TEDDY IS 8 TOS OA NAME oe aa re ASC ‘e, oer) [ore loz Ibe" Rv a ONE DSADVANTAGE IS THAT yOu NEED THE FULL SENTENCE BEFORE Yok BESIN- OT SUMABLE FR UVE SEGA RED PP os, Pi om, ae " Ck scfovoniezQEQUENCE MODELS - COURSERA = WORDEMBEAYNES MAN 1s To WOMAN 4S KING 1S To QUEEN ‘PROBLEM = THE ONE REPR Og, OF ARE HK ND IND ABUTS RetaTonSAE TO. Gp ORANEE (BANE SUS SE Sia= — [SOUUTION: CRERTE A MATEIK OF TFERIMRES TO DESCRIBE THE WORDS WORD EMBEDING ae ona aS fon tie ae oo ! Css INV REALITY -THE FEATURES ACE [Eaeniep « Nor 4 Stasitre> AS GENDER/ASE oman Lan word emeetss 19 noes THAT ANORANEE FARMER (Sr PERSON 23 Sin nane + Aone une NIE SA Sh RD EMBEDENGS TRABD ‘bs EG CET EME TERE UNAM AOS "Coun ariee) DX. MAN IS To WoMANAS RING STD 2 eq gNea Ba Sea a! om 0 0 oni ox 0.01 NANaona roe? NDGA)? . ARG MAXIE Pastrana) Sontay= _ Hak, ey BE [EARNING WORD EMBEDDINGS ‘HOlo.D> WE LEAH TRE EMBEADING MATRIX E?2 Det USING & NEWEAL LANG MEDEL 1 poawr A GUASS oF ounce f WE GAN USE DFFERENT CONTEXTS THAN HE LACT 4 WORS LAST os Lonees LETT SupGRAM Last ones SHPGRAM ~Refkk Pihces < RAND wb Ex 5 00K INEA2: SKIPSORAMS — WDRDAVEL TANT A GLASS OF ORAME DUKE To Go ALON ML CREAL LE RAIDON CONTE AREET Pas (art 5 ors) cree o sae a ean £2) a TBs [sve lmeuranonduly ExperssVe ATW CRN HPTIMTAE Bs Taig he HERBAL SHIM sine IDEAS NEGATIVE SyapUN A.PICE A ONTEAT/ TARGET faye. AS A FOSITE EXAMPLE ZIG A FEO NEG EXAMPLES. CATT + RANDOM. Ineo BML antirERS v 8 Sie oe HIS IS OWA CHAIR, | CNN —,> RN _— Ack RE MOST UKE PU a mn ye WE Dont ANT A Rabon GENERATED SENTENCE (ETA. SMETINES BAD) CE we SOMETIMES INSTEAD We WANT To. MAKIMIEE AEWA PLY omit Lx) IDEAS USE GREEDY SEARCH PICK THE WORD eT THE BEST PeveAsILITY REPEAT WNL SORE BIGRAMS Cc anes 1% pewaury {HORE al mie % IS CALLUUATED SINS A SMALL NEWRAL NETWORK 5-575) gc gn ee”D ao Ai? ® a) ‘Fauna: THERE ISA CO ON BE AT Rene > HE aie se. BUR CORE & fo PROBLEM: IDs a AT DEA: Ce 6 He 1S ee (6oeS| SoRR. tt pone» 000 NPS Bur TN TRE REAL TRANSUPTION g| | EoReor | a ‘Ly 2p OUTTUTS: 3} \_ YaonD Be Che THE WE THE THE THE THE) “THE THE, ISE Cre (OST eer, ~ GREE a mas “Re hatte === 998. fie a, DDE: en PRA NOD TE [SQMTORE] TeANSUNTE 4 Lime Ar COMRPSE REPO OURS NOT'SEP By BUNK tigen naroc denier | AINE WOW NSE | eRe BRAD SETECHON SENTENCE sore: 273 “SUR SENTENLE AS CONTA Ome, yO [oeaE) Ce CAT THE CAT ON THE MAT m ao VEN | A eR eam te ¢ $3 | dug Eabline au lly v wuss) DE VISE arti EN we 2 @restoventez QEQUENCE MODELS - COURSERA