* ML Fundamentals * freitate machine td Avility of tote gence human Aotificlal Intelligence Algos to incom pomate Mptelligenc Into machine by automatically learning feom data- Algos that mimic human woain to Incosposate into machines Inte Nigence al_ Intelligences study of how te E=AD can ao tnings computess, so that compu tess MUMTEREHernans © ca, 4°. Axvtificiat + Inteliigence ae aie oae ® a — Bauman /non~ natural Cae to a mode thing. think xo leans © Machine leasning (ML) ! Enable computens automatically From past alata, it can impwove by gaining mose data. its pesfowmance Mt uses algos feos building math models & gedictions using histo~ical data) info. eatin.”* Applications of Odrmage @Epeech Recagnition @Fecammendation Systems @¥eaud vetection @®sert Deiving Cass @ Meaicat Recagnitton Diagnosis up @ Pata © Psivacy @stock Masket teacing *Types of ML * Limitations of ML? Oodata Quality Quantity @ Bias & faioness @® ovesfitting & vodesfitt ing & Secustty © Intespoetabiiity Unsupes vised Reinfosce ment Calculate Outcome. Discoves pattesHs| Lease sesies of action Leasn patteon bet” f/p & theis labels. Find pattesn in tp& divide fn classes. Frod best twewasd foo actens. Buila & twain, then test model. classifica® & Regaession- *Aigos— OUineas Regs @vecision tee @xnnwn O©svm Q@rerdem Fowest: : Betiass Bulla & twain, then test mode}. Train & test mode} simultaneously, Cluste-sing & Assoctation e Algo 2 O k-wean Clustesing @ Agglome-sative Explosqtion 4 Exploitation, sAlgos — © Q-Learsning @ sarsa @ Deep @ cet work Ble © Exomples— Ocust Segmentation. @Featuse Elicitation. @Tawsget Masket- * Examples— @osives less cam @seF- navigating Cleaness.* Machine Leasning workflow — é Coltection Reston Handling categowical } C ieee) aid | eee Model l toalving | Evaluate e Medel | Data al —— | _» Gathes Foor Aiffesent sousces IWke database, file / senso. Cleaning & fowmatting saw data. = data to tsansfosm into numbews. Accoading to? . - Pesfoomance( Model sesult Quality) Pevfosmance' - Expl ity CEasy to inteopeed model) - Dataset size (Poocess& synthesize data) Lime & cost (fom model twaining) = es -stast with existing dato. —1—* - Anatyee data te find pattemns: -Make poedictions: Evaluate peofosreance of model. -Re Jon Mean- sq-eesos, mean- absolute. eosow, 0 2-Scowe, poot_mse. - classification: Aecusacy, Precision, Recall. Find the best model using mange of values fos hypesparsams. Deploy model to the cloud, to wun model oves the cloud.Oconfusioe Matoix — petatied vrew of model's ee petfocemance by showing cpunts of Teue ve (TP), True —veCTN), False +ve CrP) § False peedictens- Sua erhs Actual aeve False aeue | True +ve Falee ave cre) cer) Psedicted zx! False -ve Tae “ve Fals a Cerny tn) ® Accusacy— (mMeasuwe model peofowmance), LeMeasuce how often model is cowwect- - coemect TeaTtN Predictions oy tN + Fe +N a Se Total predictions @ classification Exsow— (Misclassification Fate). ) Classitication Fesos —— emeasus pow often classifee fs fo comcect - Eusos S- Accusacy | @® Psecision — How many predicted Teue ant comoectan- aes Ur rew ea ane model's +ve poedictions at: ane TR+FE@ Reece How many actual Tue eowwectly euedicted. ada TRHEN Recall = © F4- seowe — Evaluate ovesall pesfosmance of meadel. \easmonic mean of Peectsion § Recall. = _2* Peecision * Recall! Poecision + Recall @ Ave- Roe Cusve — Ly anaryze classifies at aiffesent theesho\d values. * ROC— Receives Opesating chawectenistias * AUC— Avea Undes cusve of ROC Ploty w Roe cusve — Pwovobility gwaph to show the GE CE 3 pesfoomance of classification model at Aiffewert theeshold levels. 0 4 — Pewtect Classite© Fos Rego ssto © Mean Absolute Exasow — SS solute Ens os W Analyze joss oveo whole dataset- \w Ewscos: aiffewence bet? poedicted & actual values: Mae = 3S | Sesed fes @ Mean Squased Esc ow — Mest commonly used. “Used tO calculate los Le Find ewwoe £ squase ft, theo find average oves the whole dataset: co) Le Alsaqys tves be2 we Squasce values: Le Small MSE, betties pestoomance: —_— 1 " Mee 2° > Cdened = Yastua)™ feos © Rect Mean Squaved Sewers — Evaluate pesfowmance: WIndicates how much data points spredd around the best \ine- Le Std deviation of MSE, \ylowes MSE-Folnts close to best Wnre. aust = suse = | S (Yevedé Yactuat) ~ a N @ mean Absolute Petcentage E-sswos C MAPE) — e Expsess eosors in Ceoms of pewcentage. _ Smaties pesfowmance => bettes pesfosmance. B (Cesea ~ Sco )) tes Yactual ee 100© Explosatose Data Analysis CEMA): — W Methods to study £ explose wecowd sets to apprehend thels poedominant tvalts, discoves pattesns, locate outtlews, & Mdentify Erne re eae between variables: (2OeAISSO ESE @ Data cleaning! Handte missing values, duplicates , Outiiess £ handle categorical data. @ Data Visualization! visual techniques weproes ent @ statistics gwaphically. Histoqoams, box piets, seattes plots, iWne plots , hest maps £ baw chasts +t° identify styles, teends & weiationships within Facts: @ Feature Engineesing: contain cating, vowmali zation, broning, encoding vawables. (@yeemeeiacion * Relationships sien siaiacetes wselationships fosselation & Relationsh'ps! & aependencies scattes- plots & pass- tabulations. bet” vawiables. Cowwelaticn analysis, © Data _segmentation— Divide Info inte segments based totally on suse standasds /toq'ts- ®@ Hypothesis Genes ation Genewatiog hypotheses/ studies queshons based of poeliminacy explesaton ef data. @ Data Quality Assessment— Pesmits assessing nice & welfariity of the tofe. “30 olve checking wecosd integutty, consistency & toto suitable fow analysis: to make@O_EDda Imple mentation using Pytl ° Get Info about dataset — >>> df. shape Sle > Ceows, cols) >>> af. descutbe () p> Give wesults of basic statighcal computations on the numewc CONUmNS. >> df. infedd C]p = tofowmation ( Featust name, me-null oN Og dtype) of dataPoame: * Changing Dtype (Object —» datetime) — o> af [mate] = pd. to_ datetime (Caf ['nate'J) ° Unique elements in dataset — wr AF. nunique C) C/p > ne. of volque elements in each column, issing Values — @Hendting ™ ¢ Isnulo— Check any missing values fn dataset. >> af. isnullé). sumo) > fetusn sum of np-nanC Nett) values*fillnacy— FIN value at me NULL places. se af [reclumn']. filleat value, fnplace= Teue) «seplace)— Used te place values in dotasct- >>> af [teal] = af Tteol!]. weplace Cop.0an> value) sawopnac)— seop Tecouds with Nuit waluss: Soo af [teel].acopna (axts= Or how= tangy") o duplicatedC) — Check ?f aupltcates proresent tn dataset: e >>> af. duplicated © Ojp> Retusn BGoolean( False > Deigind! » fos cach cow *n Dataset. taue > Duplicate) apy AF. duplicated). sume) C/p > Retuso total numbers of duphcate wows. 0 Amp- duplicates — 2eOP the duplicate cows. >>> af. acop-duplicates( keep = ‘Host’, inplace = Taue) Olp> Keep Hust copy & verove all Others uplicotes. 6 © Cheek ———— +. of Values 'n each. Column — Rivaiiemecnunesc) — Retusn count of each unique Walle = count SC) value tn the column “ord Af [i eot'J. vatue- counts)a ® data Encoding — W Encode categowical data fnto oumesical values. \y One= het _Eneeding / Lavel Encoding: : Enea >> Feom ckleacn.prepsocessing fmpeet Co eer al >> encodes = Label Encodes) >> af [rcoi!J = encedes. Fit-twansfosm Caf [*eol'J) L/P> Assign num to each categorey, stact'rg Foom 0. Visual zation *- Lwavatyre data jin the fesm of goaphs/ maps 5 easy uncdesstand toends /pattesns. OHwistogsam — count of oumewe values pawresent in wequias Intervals: >>> svs-bistplot (x = ‘coll, data = af) ® Boxplet — >>> SNS. bOxplot Cx = 'cols', y= 'cal2', data=daFf) © gaisplotO- Faiewise MNatstbutions Mm a dataset. 77 SN8. paisplot CAF, hue = ‘col’, herght = 2)© var Alin. °O. \ outites = Out of ange of nowmal obsects: Ww outttes detection analysis cla outties mining: & many ways te detect, but same way for cermbyat. * Outlies Detection ? — ss pprgns. box plot CX = tect t', datas AF) « Te@R (Sntew Quactile Range): ° Removing Outivess: Au <> ‘ Ww most commonly ased weve _ oot + Uppes Ext seme kechnique: i __ ants is Outives base Va tue TOR = Quaotile 3— Quecties) Uppes @uastile a 7 Os enp- pewcentiie CafLteet J, an So method a et eee 3 = op. pescentiie Cat L'eel ms, methad = kgntetpoint') Lowes Quastite sas I@R= @3-e@e4 ws Extwceme| + Uppes& Lowes bound: uppes= QG+ 1-5 *IQKR lowes= Q4- t-3* IAQK ae Lowe! outsier} i then, uae uppes & lowes value te «move cutl’ess. uppes-val = op.whewe ( ei Sasi a) ee ae acne) Joweo_val = Op. whem CaF [reo <= lowes) diRomeve OutWews Of .acop( uppes-vallo], inptace = teve) JF. dwsop ( lowew-va) Co}, inplace = tove)