RITEC mI TEC DS, ML, DL, NLP & Al With PYTHON — Classroom Notes RR ITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899166, 8790998182RITEC ‘#883 Level 07 of 08 : Basic Python Hit. 1. Introduction. 2 Data Types and Variables 3 List, 14 4. Operators 7 5. Control Flows, 19 5a if..elif..elif.. else 19 52 While Loop: 20 53. Fortoop: a #HHt Level 02 of 08: Advanced python ###t 25 6. Functions, Methods, Modules & Packages 25 6. Functions 25 6.21.1 System defined Functions 25 6.1.2 User Detined Functions (UDF) 26 613° Tuple 28 6.14 Scope af objects in functions 2» 6.15 Scope of objects in functions / Nested Functions 29 6.16 Defaultand flexible arguments 32 62 Methods 33 63 Modules 35 64 Packages. 37 6.4.1 User Defined Packapes 37 64.2 System defined Packages 79 7 Dictionary 3 & Lambda functions 46 9. Syntax étrorsand Exceptions 10. Iterables & Iterators St 14. _ List comprehensions 57 412, Generators. 59 #t# Level 03 of 08: Python Packages for Data Science ### 63 ———— 2 RRITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899166, 8790998182RITEC 13. NumPy package. 63 14. Pandas (powerful Python data analysis toolkit) 67 141 Introduction ”7 142 Slicing Dataframe. 67 343 Filtering Dataframe 69 144 Transforming Dataframe 70 145 Advanced indexing, n JA.8 Stackand unstack ™ 147 Groupby and aggregations 16 15. Matplotlib data visualization 78 16. — Seaborn data visualization 86 17. Bokeh data visualization. a7 19. Import Doto from Flat Files. 89 20. Import Data from Excel Files, 94 21. Import SAS and STATA Files: 22. Import HDFS Files 98 23. Inoport from Relational Database (Ex: SQLite). 99 24, Import web data 104 (25. — Import using urilib and requests pockoges = 106 26. Read HTML with Beautifulsoup package. 107 27. Import JSON File 109 (28. Mavie and Wikipedia APIs 110 29. Twitter API 42 30. Cleaning Data (ETL) 114 30.1 Melt() data 35 30.2 Pivot (un-melting data) 36 303° Concatenating 30.4 Merge/toins ——___ —_____ 0 22 30.5 Data Types Conversion, ——— 3 wu RR ITEC #209, Nilagieé Block, Adithya Encfave, Ameerpet @8374899166, 87RITEC 30.6 Regular expression operations 324 307 Dropping duplicate data 325 30.8 Filling missing data, 126 30.9 Testing with asserts, a7 31. Time Series Analysis 128 #888 Level 04 of 08: Machine Learning Models ### 338 32. Machine learning 134 32.1 Supervised learning 135 32.1.1 —_knearest neighbours algorithm 138 32:11. Introduction 138 32.112. Measuring model performance. 140 32.113. Hyper parameter tuning with GridsearchCv 140 32.12 Unear Models__ 143 3212.1. Logistic regression 143 32.2.2. Understanding Classifition Reports, 32.123. Confusion matrix and ROC Curve 14s 3212.6 AUC computation, 17 32.125. Myperparameter tuning with GrdSearexV_— na 32.1.2.6. Linear regression 19 32.127. Ridge and lasso regression 160 32.1.3 Support vector Machines (SVM) 161 : 322.31. Introduction at 32.3.2. Support Vectors Classification (SVE) 162 > 32.133, Tune parameters of SVM(SVC) 32.134. Support Vectors Regression(SVR} } 32.14 Precprocessing of machine learning data 164 g2aaa, Outs y 32.1.4.2. Working with categorical features 166 2 32.1.4.3. Regression with categorical features using ridge algorithms 167 32.144. — Handling missing data 169 32.1.1 MLPipeline (Putting it all together). wm 32:12 Tree Based Models as 3212.1. Decision Tree for Classification 173 5 32.1.2.2. _LogisticRegression Vs Decision Tree Classification, 175, 3 32.123, Information Gain (16) 176 . 32.12.31, Entropy and Information Gain ww _ 321232. Gini index 178 32.13 Decision Tree For Regression 181 J 32.14 Lingar regression vs regression tree 183 32.2 Unsupervised Leaning, 14 J 322.1 kemeans dustering 185, 3 we RR ITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899166, 8790998182 t fsRITEC sft Level 05 of 08: Deep Learning ### 189 33. Deeplearning 189 33.1 Introduction 189 33.2 Forward propagation 190 33.3 Activation functions. 191 334 Deeper networks 193 33.5 Need for optimization 195 336 Gradient descent 19 33.7 Backpropagation 202, 32.8 Creating keras Regression Model 208 33.9 Creating keras Classification Models 208 33.10 Using models 208 33.11 Understanding Model Optimization 210 33.12 Model validation 212 32:13 Model capacity 27 #04 Level 06 of 06: Project on Deop Learning ##it 2139 34. Project using keras and tensorflow 219 35. — Convolutional Neural Networks(CNN). (226 ‘tiHt Level 07 of 03: NLU/ NLP/ Text Analytics/ Text Mining ###__ 230 36. Natural Language Understanding/Processing (NLU/P} 230 36.1 Introduction 230 36.2 Regular Expressions 2a 363. Tokenization 233 36.4 Advanced tokenization with regex 234 36.5 Charting word length with nltk 237 36.6 Word counts with bag of words 238 36.7 Text pre-processing 239 36.8 Gensim 2a 36.9 Tha with gensim. 243 36.10 Named Entity Recogrition. 246 EE es 5 IRR ITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @82 9186, 8790998182RITEC 36.11 Introduction to SpaCy 246 36.12 Multilingual NER with polyglot 248 36.13 Building a “fake news" classifier — 249 36.14 Dialog Flow, 255 36.15 RASANLU 256 #88 Level 08 of 08: Projects on NLU/NLP Hi 257 37. ° Introduction, 257 38. EchoBot 257 39. ChitChat Bot. 258 40. Text Munging with regular expressions. 260 41. Understanding intents and entities. 262 42. Word vectors. 267 43, Intents and classification _. 270 44, Entity Extraction 21 45, Robust MLU with Rass. 274 46, Building a virtual assistant. 276 46.1 Access data from salite with parameters 26 46.2 Exploring a DB with natural language. 9 46.3 Incremental slot filing and negation 282 47. Dialogue, 289 47.1. srateful bots 289 47.2 _ asking questions & queuing answers 298 48. Heading 294 49. Heading 294 RRITEC #206, Nilagir Block, Adithya Enclave, Ameerpet @8374899166, 8790998182RITEC ‘##H# Level 01 of 08 : Basic Python ### o uido van Rossum is a Dutch(Europe) programmer who is best known as the ‘author of the Python programming language 2. Python is a programming language that lets you work quickly and integrate systems more effectively. 3. Use python General Purpose: build anything Q se Python for. Web Development: (janz2, Pyramid. Gomle , Tecra, Hask. webapy GUI Development: thine: . PyGO% + PHQE, Prside Kivy, wsPython Gcientific and Numeric. =, Software Development: 2uilbot , irc, Roundup ‘System Administration: Ansible , Sait , Openstack 4. Python is open Source! Free! Free! Free! 5. Python mainly has two version python 2.x and python 3.x. For more information refer htt thon ora/downloads/ 6. Download anaconda software https://www.anaconda com/downloads and install it a. Anaconda software = python software + around 1400 python packages of data Science 7. Spyder is a powerful interactive development enwvironment(IDE) for python programing language, 8. Observe spyder tour a. Start > anaconda > spyder > help > Interactive tours > wedhicion tous 9. Observe Jupyter IDE rn 7 RR ITEC #209, Nilagiri Block, Adithya Enclave, . ..cerpet @8374899166, 8790998182RITEC a. Start > anaconda >Jupyter i. It is aweb based IDE,usefully to develop nutebooks i. Observe it more we will see in next lessons RR ITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899166, 87909981822. Data Types and Variables 4. Exercise 1: Python as calculator RITEC a. Open Start > All Programs > Anaconda3 > Jupyter Notebook > Right side of the browser > dick on new > under notebook elick on python3 a Upteae [News Python 3 In [2]: 243 in [4]: 17/3 out[a}: 5.66sss6se66s6667 In [5]: 17//3 ourls}: 5 am [6]: 17%3 outte}: 2 In [7]: 582 — ourl7}: 28 2. We can access previous step result using underscore" _" RR ITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899166, 8790998182RITEC In [7]: 5 #82 our(7}: 25 In (8]: _+3 our[s]: 28 3. Save Jupyter notebook 4, Saved notebooks are available in the path Isers\Hi\ 5. Let us see some Basic data types a. Numbers (int, float and complex) b. Strings . Bool 4. Lists e. Tuples f. Dictionary 2: Define some datatypes a. Use type() function to know the variable data type # Define some sample data types # 01.Defining integor savings = 1000 print(savings) # 02, Defining float interest = 4.5 Print(interest) # 03. Defining complex a_complex = 1+2J a_complex 10 RR ITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899166, 8790998182RITEC ‘#04. Defining String dese = “compound interest” print(desc) #05. Defining Boolean ‘am_j_good=True print(am_i_good) # 06. How to know data type of a given object? Print(type(savings}) print{ype(interest)) Print(type(a_complex)) Print(type(dese)) Printtype(am_i_good)) 7. Exercise 3: Convert one data type into other datatype a. Convert one data type into other using functions int(),str()float(),complex() and bool) #01. Convert i a=10 ‘ype(a) at = sta) type(at) #02. Convert float into str b=10.5 type(b) bt = stb) type(bt) uw RR ITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899166, 87909 © .37; RI TEC #03,Use cas calculating simple interest savings = 100 interest=12 total_amount = savings + 100 * interest * 2/100 # convert total amount as sting and print user friendly info print("l started with $+ str(savings) + " and now have $" + str(total_amount) + *. ‘Awesome!") # 04, Convert string into float pi_string = "3.1415926" print{type(pi_string)) #05. Convert pi_string into float: pi_float pi_float = float(pi_string) print(type(pi_fioat)) @. Exercise 4: Read characters from string [# 01. Create a string #03. print h word[3} #04, print slice pyth ‘word[0:4} #05. print slice starting to t word{:3] # 06. Fancy reading RRITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899166, 8790998182for tin {0,3,5) Printwordi) 9. Mutable vs Immutable Objects: A mutable object can be changed after it's created, and an immutable object can't 10. String is immutable In (44}: worde! THON In [22]: word ouea2):syTHoN ye [13]: weraf Typetrrer Feacebeck (most recent all last) ‘python input-12-bsa7a2se6eb9> fa o sty a woraid= PP Typetrror: 'sen" object does not support item assignment 11. Refer https://docs. /3/tutorial/introduction. htm! 12. More about numbers refer hitps:/idocs.python,ora/itutoriaVintroduction htmifnumbers, 13, More about Strings refer hitps:/ ca/3Mtutoriali p.himniéstrings ° 1B RRITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374P. > 66, 8790998182RITEC 3. List Python List fa, b, ¢] 2. The most versatile (flexible) is the list, which can be written as a list of comma-separated values (items) between square brackets. 3. Lists might contain items of diferent types, but usually the items all have the same type. 4. Exercise 1: lefine a list with same/different data types # Define alist with same data types rritec_emp_sal={1000,2000,3000,4000] print(ritec_emp_sal) print(type(tritec_emp_sal)) # Define a list with different data types rritec_emp_name_s: 'ram*,1000,"nancy",2000,”robert",3000,"rabson”, 4000] print(ritec_emp_name_sal) # A list may contain other list (nested list) rritec_emp_name_sal1='ram*,1000),{"nancy",2000},;‘robert",3000),["rabson".4000]} print(rritec_emp_name_salt) printtype(ritec_emp_name_salt)) : 5. Exercise {ead list elements(Slicing and dicing) ‘# Read list values (Slicing and dicing) print(rritec_emp_name_sal[0)) print(rritec_emp_name_sal[-1]) - print(rritec_emp_name_sal[0:4]) ~ print(rritec_emp_name_sall:4]) i print(ritec_emp_name_s™4:}) print(eritec_emp_name_sal{0}(0)) 14 : RR ITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899166, 8790998182 ‘RITEC print(rritec_emp_name_sali[Ol-1]) 6. Exercise 3: Manipulating lists a. Modify elements ,Add elements and Delete elements of list bb. List is mutable # Changing list value rritec_emp_name_sal[0}="Ram Reddy’ rritec_emp_name_sall2:4]=[.wilis’,2100] print(ritec_emp_name_sal) # Adding Elements rritec_emp_name_sal=ritec_emp_name_sal + ['Helen’.5000] print(ritec_emp_name_sal) # Deleting Elements del(rritec_emp_name_sal[2:4]) print{rritec_emp_name_sal) 7. Exercise 4: Observe list assignments memory allocations i# Nood to observe 1 x = [1000,2000,3000} y 2 = list(x) vyfo]=1001 print(y);print(x) # Need to observe 2 }000,2000,3000}, 2 list(x) 2{1] = 2001 15 RR ITEC #209, Nilagiri Block, Adithya Enclave, Ame ....c w#374899166, 8790998182RITEC prini(z)printgx) 8. Refer 1 of 2; https:fdocs python ora/SitutoriaVintroduction.htmidlists 9. time permits ,refer 2 of 2: httes:/idocs,python.ora/3utorialidatastructures. html 16 RRITEC #209, Nuagiri Block, Adithya Enclave, Ameerpet @8374899166, 8790998182RITEC 4. Operators 4. Operators 1 of 3: Comparison Operators are (<, >, >I) ‘8 Py22_1_Compasison, Operators 1 print("ex. Comparison of escteans") 2 print(True == False) 3 print(True I= False) 4 5 print("@2. Comparison of in 6 print(-5 * 15 l= 75) 7 prine(2 <= (1 + 1)) 3 9 3. Comparison of str 1 t= "python") u Python” te python") voce 2 |. carital print("2s. print (True print("e5. Conpare ~unpy U2 import nunpy as_np arks = np.array( {10,28, 38,49,50,68, print(marks>50) print (marks[marks>50]) 2. Operators 2 of 3: Boolean Operators (and, or, not) © °Py22_2 Boolean, Operators 4 print("@2. AND Boolean operscon") 2 x=6 3 print(x > 5 and x < 15) 4 print("@2. Of Bootean opers: 5 ys5 S print(y <7 or y > 33) 8 8 2 i vy print("es. NOT Boolean operszor) Print (not True) print (not False) print ("ed. working with ns import nunpy as np . 12 marks = np.array({10,20,30, 48,58, 60,70,58, 585 120}) 13 print (marks>5@ and marks<80) 2, amray is it sorking") 7 RR ITEC #209, Nilagiri Block, Adithya C....ave, Ameerpet @8374899166, 8790998182: RITEC 3. Operators 3 of 3: Numpy Boolean Operators (logical_and(), logical_or(), logical_not()) 1B *Py22_3 Numpy Boolean Operators : print("e1. Logicat_and() Boolean operator") inport nunpy as np marks = np.array([10,20,30,40,50,60,78,88,9¢, print (np. logical_andjnarks>5,marks<80) ) print("@2. Logical_or() Boclean operator") marks = np.array([18,28,30,48,50,60, 78, 80,90, print (np. logical_or(marks==50,marks>76) ) print("e3. Logical_not() Boolean operator") marks =.np.array(({18,20,38,40,50, 60, 76;2%/39,200]) print(np.1ogical_not(marks==50)) 4, Filtering data frame using operators. © y22.5 Filtering Dataframe = print("a1. Import csv file as pandas datazrane import pandas as pd brics = pd.read_esv("trics.csv", index_col = @) print (brics) priat("22. observe area column") print(bries["area"]) > nea"] 08 9 print(“e3, filter data area 28“) 19 print(brics[brics[*crec"} > 8]) 12 print("ed. filter data using boctean operator") 13 import nuwpy as np 14 print(brics[np.logical_and(brics[“area'f] > 8, brics{"area"] < 10)}) | - ——— 18 RRITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899166, 8790998182- RITEC 5. Control Flows. 54 if... elif... elif... else 1. Conditional Statements (If, elif, else), observe below two examples. int(input(*Please enter an integer: *)) ifx<0: print(‘Negative Number’) elif x a mine | ase | | print(Positive Number’) Py22_4. Conditional Statements E bees if2%2 i 5 print( elifz %3 5 print( 6 else: by 2") is divisible by 3") print(“z is neither divisible by FHP my 3°) emeeaa2 1) 2. Anif «+ elif ... sequence is a substitute for the switch or case statements found in other languages. (example switch in R language) eee 19 RR ITEC #209, Nilagiri P asdithya Enclave, Ameerpet @8374899166, 8790998182RITEC 1. The while loop is like a repeated if statement. The code is executed over and over again, as long as the condition is true. 5.2 While Loop: 2. Exercise 1: What is the output of below code? x Inttial Value whilex<4: Condition rint(x) EXPresstory Reis ialpomentads 3. Exer ise 2: write Fibonacci series up to 1000 a b= 07 while b < 1000: print(b, end=",") a, b= b, asd - eee 4, Exercise 3: Understand Break Statement a. With the break statement we can stop the loop even if the while condition is tue W Exit the loop when i is 3 isa while i < 6: print (4) 7 if ies break itea 5. Exercise 4: Understand Continue Statement a. With the continue statement we the next iteration 1 stop the current iteration, and continue with # Continue to the next iteration iis 3: — ize while i < 6: | deed ifa RR ITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899166, 8790998182L print) 5.3 ForLoop: =| Python's for statement iterates over the items of any sequence (a list / a string /a dictionary ), in the order that they appear in the sequence For each value in objectilist, Dictionary...etc) , execute expression Exercise 2: Read list of items? Fawity hedghes = (3.73, 1:68, 2 for height in family_heights : 5 print(heggnt) Exercise 3: Read list of items with indexes? y23_3 For Loop with index and,vokies = family_heights = (2.73, 1.68, 1.71, 2.89] for index height in enunerate(fanily heights) print(“ingex “+ sen(index) +“: " Meerqheight)) Exercise 4: Read list of lst items? [B *by23.a Far Loop _Ust of ats Exercise 5: For loop on dictionary? 1. The items() method is useful to loop over a dictionary RRITEC #” ©, Nuagiri Block, Adithya Enclave, Ameerpet @9374899166, 8790998182 2[B--Py23 5 For Loop.on Dictionary 4 for key,value in world. items() : print("The Population of + str(key) + * is + ste(value)) 5 6 2 Exercise 6: For loop on numpy array? 1. 1D Numpy array, looping over all elements can be as simple as list looping 1B *Py23_6 For Loop on NumPy_artay 1Dimension pert racuinat mogsles import numpy as np he in np_fanily_heights print (height) | 2. A2D array is built up of multiple 1D arrays. To explicitly iterate over all separate ‘elements of a multi-dimensional array, you'll need the syntax: for x in np.nditer(my_array} = 1 2 3 cee ty catber 4 np_family_heights = np.array([1.73, 1.68, 1.71, 1.89]) 5 5 7 18 323.7. For.Loop_on NumPy oy 2 Deensons a reae: import nunpy as np np_fanily heights = np-array({1.73, 1.68, 1.72, 1.89]) np_famity_weights = np.array([78,-60, 65, 80}) 2 3 4 5 é 7 8 8 ht. in np-nditer(np_tanily. heights weights) : '3__print (height) Le ——————— 22 RR ITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899166, 8790998182RITEC Exercise 7: For loop on pandas dataframe? 13 -Fy23.8 For Loop. on pandas dataiame * 1 import pandas as pd 2 bries = pd.read_cev("brics.csv", in 3 print("rerer* 21. Column Names’ *>™ 4 for val in bries : 5 print(val) 6 print ("rete @2 ALL rows TeTNTT+") 7 3 r print(row) print(*7=t root" 93 Selected Columns **>e=77"=") 33 For lab, row in beics.iterrows() # 42 print (lab +“: " + rowf"capital")) 3B print("**=*7*** 98 ddd Column ta result *srt=ere=") 44 for ab, row in brics.iterrows() iS _bri¢s.loc[lab, "nome_Lengt”] = len(row[ “country” ] ) 16 print(brics) 7 print("™ ther methaa to add Coluan to <: 48 brics[ "none, brics{ “country”}. apply (len) 43 prine(brics Refer document: hi Cs. 1 rialcontrolflow.htmlit 23 "EC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899 166, 8790998182RITEC 24 RRITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899166, 8790998182RITEC ‘#8 Level 02 of 08: Advanced python ### 6. Functions, Methods, Modules & Packages 61 Functions 61.1 System defined Functions 1. Piece of reusable code ,Solves particular task 2. Exercise 1: understanding predefined functions (str, type, len, max...etc) Function {18, 20, 30, 49) 3 True 3 datatype of var1 is :’ + str(type(vara))) 7 Length of the voriasie is “+ ste(2en(vard))) 3 Takes of 8 out2 = int(var2) eITEO 18 print(‘The moximue the List elerents is : ‘+ SeA(max(Var1))) 3. Exercise 2: understanding sorted functions rv! full = first + second full_sorted sorted(full, revers tout Full print(ful1_sorted) 4, Refer standard document of python s:lidocs. python.or¢ i .htmiitdefining-tunctions ——— 25 RR ITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899166, 87909981826.1.2 User Defined Functions (UDFs) RI TEC 1. Though we have lot of built-in functions sometimes, we need to write user defined functions Exercise 1: Function without arguments thout arguments 1h tetane 44 2 def welcome(): Deri: Function with one arguments I ¥y24_2 Fanction_with_one argument 1 We Funsiies Meath def square(value): 4 6 square(16) Exercise 3: Function with one arguments and return vaiue 2 -Py24 3, Function, with one_argument.with tum valve 1 2 func 2-def square(value) 3 3 return value ** 2 5 = call the furction and assign va? 6 nua = square(3) 7 print(num) Exercise 4: Function with Docstrings 1.Doestrings describe what your function does 2.Serve as documentation for your function 3.Placed in the immediate line after the function header 4.1n between triple double quotes “™ or single quotes EY 26 RR ITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899166, 8790998182RITEC 2 py24_4 Function with doestings "print the square of a given vatue 3 a metine 5 print(value ** 2) 7 square(1@) - nial Exercise 5: Function with multiple arguments ng fu der 2-def square (valued, value2): v"Raise valuei to the power of value2"** new_value = valuel ** value2 return nex_value ————— 27 RR ITEC #209, Nifagiri Block, Adithya Enclave, Ameerpet @8374899166, 8790998182rRITEC 61.3 Tuple Exe © 6: working with tuple 1.Tuples needed to return multiple values by a function 2.Tuples are Like a list - can contain multiple values ‘3.Tuples are immutable - can't modify values! ‘even_nlms = (2,4,6,8) even_funs[1]=9. = "988 4-Tuples Constructed using parentheses () 2B Py24.6.tuple # print("s9s¥9 ai Define tuple print (even_nuns[o}) 2 even_nums = (2,4,6,8) 3 print(""=*=~ 22. know the Dato tice 3 even_nums)) 5 Map tuple items to voriaetes *-*+") 6 nuns 7 print( + str(a)) ‘8 print("b = "+ str(b)) 5 prine("e = "+ str(e)) 18 print("d = "+ str(d)) na AL print( * @4. Access tuple elements Like you do with siste Exercise 7: Function with multiple return values [3 py24_7 Function, with multiple, return valves p define fonction non Ydet sth(valuel,value2): 3 “"Raise valuet to the power of value 2 ond vice versa."" 5 new_valuel = value ** valuez 6 —newivalue2 = value? ** valued 7 new_tuple = (new_valuet,new_value2) 3 return new_tuple 9s cell the Suresie 19 rs = raise both(10,2) BEL 2 print(rs) ° RRITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899166, 8790998182RITEC 6.1.4 Scope of objects in functions 6.1.5 Scope of objects in functions / Nested Ful 1. Variable values are searched in the order LEGB i 2. Local scope > Defined inside a funtion 3, Enclosed Scope > If a function is wrapped inside another function 4. Globat scope > Defined in the main body of a script scope > Names in the pre-defined buil-ins module (ex: len, max, min, ... etc) 6. Exercise 1: cannot access local variable globally 1 2 3 newvalue = value ** 20 § (o0u3 Vo 3 print(new_value) 5 A z square (10) Pint (new value) 7. Exercise 2: we can use same name for local and global and assign different values 18 Py25,2 Scape. of functions, ule? = 1 newvelue = 10 ¢ G.crai Variséls Bee metus sssger 4 def square(value): 5 new.value = value ** 2 wd 5 print(nen_value) 2 square(3) prine(new_vatue) | 29 RR ITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899166, 8790998182- RITEC 8. Exercise 3: searches for local variable ,if not find then searches for globally 1 Py25_3. Scope of functions, rule} = i power = 2 28 lefine 3 def square(value): new_value = value ** power print (new value) a 5 é ? square(3) 9. Exercise 4: can override global variables within the function new_value = 10 = print (new_value) def square(value): global new_value new_value = value ** 2 print (new value) square(3) print(new_value) 10. Exercise lested functions 1B +Py25_5, Nested Functicnsl * 1 def modzpluss(x1, x2, x3): 2 "™"Beturns the renainder plus 5 of three values 3 def inner(x 4 “enpeturns the renainder plus 5 of « velue 5 return xX 2+5 é 7 3 return (inner(x1), inmer(x2), inner(x3)) print moapauss(2,3,4)) 11. Exercise 6: Nested function |B Py25.6 Nested Functions2 i-def ratse_val(a): “ngezurn the inner function." def inner(x): wuRaise x to the power of n.™ raised = x "0 return raised return inner square = raise _val(2) - print(square(3)) ee 30 RRITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899166, 879099818212. Exercise 7: Nested Functions =Py25,7_Nested Functions3 ade outer(): 2 Prints the value of 2. 3 net 4 deF inner(): 5 nonlocal n 6 n=2 7 print(n) 8 inner() 9 print(n) 20 outer() iFEC uf 1. Exercise 8: Builtin understanding ‘global variable’ “def len(in_var): print(called my len() function’) 1=0 for jin in_var: les return I" def a_func(in_var): en_in_var = ien(in_var) # builtin print(Input variable is of length’, len_i \_var) a_func(Hello, World!’) RITEC 31 RR ITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899166, 8790998182RITEC 61.6 Default and flexible arguments Exercise 1: Default arguments B *Py25.8 Default Arguments © Js a-deF power(nunber, pons: poxer of parameter pow.” pow Exercise 2: Flexible arguments: “args 8 -Py25.9 Flexible Arguments strorgs © 4 def add_all("args): “Sum all values in “args togecher.” sumall = @ for num in args: sumall += num return sum_all 38 print(add_al1(1,2,3)) 11 print(add_all(1@,20, 3¢,40,50)) x Exercise 3: Flexible arguments: “*kwargs fb Py25.10_Feible Arguments large = Andef print_al1(**kwargs) “print cut Rey-volue pairs in “*kwargs." 2 3 ¢ " 4 for key, value in iwargs.itens(): 5s 6 print(key +": "+ value) y print all (studenti="Rateon”, student2="Robert”, student? ="Ram Roddy” Ets 32 RR ITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899166, 8790998182mITEC 6.2 Methods 1. Methods are Funetions that belong to particular objact 2. Based on object data type different methads available 3. Exercise 1: List object methods # Define list alist = {10,20,30,10,30,40,50,10,60,10} # Count how many times 10 repeated 2 _[istcount(10) #add 100 to list a list-append(100) a_list # add 200 In 3rd position a_list.insert(2,200) alist # remove 20 from the list 2 list remove(20) # sort items a List-sort() alist # reverse items a lstreverse() alist 4. Exercise 2: String object methods RR ITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899166, 87909981821" str = "Data Science” # Convert into uppercase a_str.upper() ‘# count number of times “ e" repeated _ a str.count("e") ——————_ 34 RRITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899166, 8790998182RITEC 63 Modules 1. A module is a file containing Python definitions and statements. 2. The file name is the module name with the suffix py appended. 3. Spyder working directory available in tool bar mo pGBCs we 4, Create a module and save into Spyder working directory as “a_module1.py" os =>e BX Fee # Define anumber a_num = 10 # define astring acstr “python # define alist a_fist =[10,20,30,40} | # define afloat eon | # define function square def square(x): : return x""2 # define function fibo def nbo{n): a,b=0,1 while b NumPy is the fundamental package for scientific computing with Python(example array) 2. Scipy > Scientific Python distributions 3. Sympy + Python library for symbolic mathematics 4. Matpiotlib > Python plotting package 5. Scikittearn > machine learning 6. Math -> Mathematical function (example pi. sin, cos ...etc) 7. xird > to work with excels, 8. keras > Deep Learning /neural networks 9. nitk > natural language too! kit (NLP,NLU) 10. rasa_nlu -> NLP,NLU 11. TensorFlow > programming environment £2.08 way Selby Worry vaplats boce endo snarl ray compere 20 sraypacage torso compute | Pion sympy pandas TP OX: cranes nba maunewes Pytron ea Serb as trative Conte mothers se 13. As we knew anaconda software = python software + basic packages required for data science 14. To see list of packages already installed in anaconda > start > all programs > ‘anaconda folder -> right click on Anaconda Prompt -> run as administrator 39 RR ITEC #209, Nilagiri Block, Adithya Enclave, Arr .. yes 28374999166, 8790998182rITEC 1 Anaconda (64-bit) {) Anaconda Navigator lupyter Notebook Reset Spyde Settings 8 Spyder 15, Type conda list, 16. Too see particular package type conda list (example see matplotlib) 17. To install package not avaitable in work area i. Right click on anaconda prompt > run as administrator ji, Cond instai -or get command from google ii, Zxample xird not came with instalation of anaconda 18. To install package not available in work area and in anaconda, 4. Right click on anaconda prompt > run as administrator ii pip install or get command from google li, Example rasa_nlu not available in anaconda 19. To see functions of a module \. Open spyder ii. Run below code in dir(math) print (x) 2. Exercise 4: lmport Package and calculate circumference and area of the circle od 40. RRITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899166, 8790998182RITEC 71 Import Ppackage = C22" r* math.pi By 1 2 3 4 5 é 7 a As math.pi* rt 2 93 Bulle cesttout 1e print("circumference of circle: "+ ste{c)) Li print(“areo of circle: “+ str(ay) <7 3. Exercise 2: Import one function from Package and calculate circumference and area of the circle 3 Py7_2_ImportOne function from Ppackage > 6 = 0.82 from math import pi are pi oo c Aspitett2 — + str(a)) 12 print(“circunfere 11 print("area of circl 4. Exercise 3: Import Package function using alias name and calculate circumference and area of the circie rele: " est) print(‘area of circle: “+ str(A)) 5. Create project using spyder 1. In spyder go to project menu -> new Project > provide name as spy_project > click on create 41 RR ITEC #209, Nilagiri Block, Adith; ._..ciave, Ameerpet @8374899166, 8790998182RITEC 2. Right click on the project > new > package > provide name as spy_package > click on ok 3, Right click on the package > click on new ~ click on module > name it as spy_module 4, Rest of the process like importing is same as above 42 RR ITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899166, 8790998182RITEC 7. Dictionary 4. Why we need dictionary datatypes? Exercise 1: Get India population using list? I convenient? (1.38,1.34,0.32) *, "USAT 5 index(“indio") print(india_pop) 2 3 4 t 5 Indie pop = countries. é pies 8 print {pop_bi22ionstindie_por]) Exercise 2: reate simple dictionary? 2 pas ‘eco Simple Dietonary world = {"ching"s1.38,"India":1,34,"uSa":0.32) print(type(world)) print (world. keys()) | 8 print(world. va2ues()) 9 Slat wre Beivs 32 print (world{fingia") 2. Observations of dictionary. a. Each key is separated from its value by a colon (:), the items/ elements are separated by commas and the whole thing is enclosed in curly braces. b. An emply dictionary without any items is written with just two curly braces, like this: 0. ©. Keys are unique within a dictionary while values may not be. 4. Syntax of dictionary: my_dict = { Skey1""valuet”, key" e. Refer: ht.psilidocs.python.org/3/tutorial/datastructures.htm! —— 43 RRITEC #209, Nilag: . ..0cx, Adithya Enclave, Ameerpet @8374899166, 8790998182RITEC Exercise 3: Insert, Update and delete dictionary items 1B Py21 3 Insert Update Delete Dictionary Items = 2 world = {“China":1.38, "Indie 1.34, "US":0.32} print(wertd) world{ "redone: worldf "éraz print( ‘Ingone print(werid) Huresta can iten 27 «0 wortdl "Tndia"}=1.39 13 print (world) 3S del world "erazit"]) 26 print(world) Exercise 4: Nested dictionaries op °31.34},\ 3 washington, o.¢°, Fe ‘ion !:8.32}}| 6 print(wortdt"Inaio’ jt ‘copitel) 8 data = { ‘capitol’: ‘rome’, ‘poputation’:59.83 } 15 wort ieely ia a here RITEL 12_print(world) 2 eenSEEeeenemeees=cneeeeeeeee eed 44 RR ITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899166, 8790998182RITEC Exercise 5: Create dataframe using dictionaries [2 924-5 Greate Dara Using Disionsies couinery_nanes = (‘United Stotes’, “austrolic’, ‘Japan’, "India", ‘Russie’, ‘vorscco", drives right = (Tue, False, False, False, Tove, True, True] eareavg_per 1000 = [909, 731, 588,18, 200, 70, 45] as pa 2 3 6 import pands 2 eters 6 2 3 my.aict = { ‘country’ zeountry_names,\ ° “arives. right :arives_right,\ 2 “cart par 1008":cars avg. pet_1000 ) 12 ears = pa.DataFrane(ny_dict) abe oer cP rsdansls 1¢ row labels = [US's 'aUS", "MP". “ZW, "RU", 16 cans.index = - ay Sstineses cost 38 print(cars) eqiet] RRITF > . 0, Nilagiri Block, Adithya Enclave, Ameerpet @8374899166, 8790996162 45RITEC 8. Lambda functions 1. Python supports the creation of anonymous functions (i.e. functions that are not bound toa name) 2. While normal functions are defined using the def keyword, in Python anonymous functions are defined using the lambda keyword. Hence, anonymous functions are also called lambda functions 3. Syntax: fambda arguments: expression 4. Lambda functions used with typical functional concepts like map), fiter() and reduce() Exercise 1: Simple lambda function 1 2 3 def ratse_to_power(x,y): 3 return x" y Eraise_to_poner(2,3) foe @raise_to_power = lambda x,y: x ™* y 1 2 raise_to_power(2,3) c Here: 4. x,y are arguments 2. x** y is body part of the function Exercise 2: lambda function with map function # define function def power(i): return i** 2 # Can you apply above function on python list? a list = [10,20,30,40] power(a list) | | #cam you apply above function on numpy array Le RRITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899166, 8730948192import numpy 45 np _np_list = np.arrayla.tist) power(a_np_list) # can you avoid above defining function and converting li map +5 = map(lambda 1 2,0 list) rs list(rs) Exercise 3: lambda function with filter function nuns= [3,6,4,12,2] def more_than_S(x): y= for i in x: if dos: y-append(i) 19 return y ers 12more_than_5(nums) 3 aes 35, ‘enums= [3,6,4,20,1] 47more_than_S = filter(lambda num: num > 18 list(more_than_5) Exercise 4: lambda function with reduce function : Find product of first 10 prime numbers. # define first 10 prime numbers (2,3,8,7,11,13,17,19,23,29) @_list_prime_num # regular way product = 1 [fer in a_list_prime_num: ee 47 R ITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899166, 8790998182# using reduce function i from functools import reduce reduce(lambda x.y:x *y,a_list_prime_num) : 48 . RR ITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899166, 8790998182-. RITEC 9, Syntax Errors and Exceptions 1. There are (at least) two buil in distinguishable kinds of errors: syntax errors and 4 exceptions. a. Syntax Errors 1. Syntax etrors, also known as parsing errors foi while Teue print( ‘Hella wortd') Console PyUnit try it 2.Name€rror > 42 Spams) 3.TypeEtror > 4.IndentationError > (type space beginning of the line ) 5.Find more errors ldocs.ovtl Sfibrarylexceptions, 2, User Defined Exceptions 49 RR ITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899 166, 8790998182RITEC a. Catch exceptions with try-except clause i. Runs the code following try If there's an exception, run the code following except b. Exercise 1: user defined exception message [B-Py27-1 rors and exceptions © a-deF sqrt(x) "Returns the square root of a number. try: print(x ** @.5) except: print(% must be an int or float") 7 print("**** of Execute vith integer values?” 8 sart(s) 9 print("=**** 02 Execute irish string volue 39 Sart "ABCO") ao ¢. Exercise 2: fink user defined exception message with built-in error codes. 2 Py27 2 Erovs_and exceptions raise dedef sqrt(): 2 Return e A ued 4 raise ValueError('x must be non-negative’) 5 else: 6 print(x ** 0.5) 7 8 print("*¥*t* @1 Execute with integer valuertes 9 sqrt(9) 10 print("**7** 92 Execute with negative volyeer4srs") 42 sqrt(-S) STS —_—————— 50 RRITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899166, 8790998182RITEC 10. Iterables & Iterators 1. Iterable 2. Iterable is an object with an associated iter() method i. Examples: lists, strings, tuples, connections. ionaries, range object and file a . Applying iter() to an iterable creates an iterator {the terable) Wtheiteratort 2. Iterator a. Produces next value with next() b. Itis simple and faster than for loop 3. Exercise 1: list iterator ¥ Create a list of strings: names ] names = [Ram’, ‘Rabson’, ‘Robert, 'Nancy’] | # Print each list item in names list using a for loop print("***"* 01 For Loop Output **""") for person in names: print(person) print("*"""* 02 Replicate for loop output with list_iterator Output *"***") # Create an iterator for names list: avoid_for | avoid_for = names.__iter_() # or iter(names) RR ITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899166, 8790998182| Print(type(avoid_for)) # Print each item from the iterator print(next(avoid_for)) print(next(avoid_for)) print{next(avoid_for)) Print(next(avoid_for)) Exercise 2: range iterator # Loop over range(S) and print the values print("****** 01 For Loop Output ****""") for num in range(S): print(num) # Create an iterator for range(5): range_iterator print("****** 02 Replicate for loop output with range_ierator Output range_iterator = iter{range(5)) rint(type(range_iterator)) # Print the values in range_iterator print(next(range_iterator)) print(next(range_iterator)) print(next(range_iterator)) print(next(range_iterator)) print(next(range_iterator)) RR ITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899166, 8790998182[a.sicter=@ arte o next(a_stt_ter) next(a_str_iter) next(a_str_iter) next(a_str_iter) next(a_str_iter) next(a_str_ter) 6. Exercise 4: Tuple iterator a tuple = (10,20,30) _tuple_iter=iter(a_tuple) next(a_tuple_ter) nnext(a_tuple_tter) next(a_tuple_iter) next(a_tuple_iter) 7. Exercise 5 fionary iterator 11ram",2:"Nancy",3/rabson"} a_dict_iter = iter(a_dict) next(a_dict_iter) next(a_dict_iter) next(a_dict_iter) nex'(a_dict_iter) 8. Exercise 6: Use iterators as function arguments ¥ Create a range abject iterator: range_object range_object = range(10, 21) print(type(range_object)) RR ITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899166, 8790998182,[a Brintthe range object print(range_object) # use above iterator object as a parameter of list ,max functions ## Create a list of integers: rans p_object_list range_object list = lis(range_object) # Print range_object_ist print(range_object_lst) +# Get the max of values: range_object_list_max range_object_list_max = max(range_object) # Print range_object_list_max print(range_object_list_max) 9. Exercise 7: enumerate function , Enumerate function returns an enumerate object that produces a sequence of tuples, and each of the tuples is an index-value pair. [ # Create a list of strings: Datas DataScience = [PYTHON",R'MATHS' STATS" ML: DL'NLU,/NLP*AT] print("**** 01, enumerate() function creates enumarate object ****") print(enumerate(DataScience)) print("**** 02. Create alist of tuples **** ") DataScience_list = ist(enumerate(DataScience)) | printtype(DataScience_tist)) | # Prine thelist of tuples print(DataScience list) | pre 03, Unpack and pnt the wl ais | for indext, valuet in enumerate(DataScience): RRITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899 166, 8790998182print(indext, valuet) print("™"""* 04. Unpiick and print with given start index “***") for index2, value? in enumerate(DataScience, start=1): priniindex2, value2) 10. Exercise 8: Zip Function 2. Zip() function creates zip object b. Zip object is an iterator of tuples. 6. Hyou want to print the values of a zip object, you can convert it into a list and then print it oftuples sstd_name = ("Ram’,"Nancy’,"Rabson") std_marks = (100,2000,300) #create zip object std_data = zip(std_no, std_name, std_marks) sté_no = (10,20,30) print("™*"** 01. Print the list of tuples create by zip function * print(type(std_data)) print(std_data) print(tist(std_data)) print("**"*" 02. Unpack the zip object and print the tuple values “""*") std_datat = zip(std_no, std_name, std_marks) for valuet, value2, value3 in std_datat print(valuet, value2, value) 11. Exercise 9: unzip function a, There is no unzip function for doing the reverse of what zip() a 55 RR ITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899166, 8790998182b. However we can use * and zip() to unzip # Create a zip object : zi 21 = zip(std_course, std_name) print((01. tuples in 21 after unpacking with *) print("21) # Re-create a zip object : 21 21 = zip(std_course, std_name) # Unzip’ the tuples in 24 by unpacking with * and zip():resultt, result result, result2 # zip(*z1) print("02. Check if unpacked tuples are equivalent to original tuples") print(resultt print(result2 | | | 56 RR ITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899166, 8790998182RITEC 11. List comprehensions 1. List comprehension is an simple way to define and create list in Python 2. In mathematics the square numbers of the natural numbers are for example created by { x2 |x € N} or the set of complex integers { (xy) |X EZ. y € 2). 3. Guido van Rossum prefers list comprehensions and does not like lambda functions 4. List comprehension is a complete substitute for the lambda function as well as the functions map(), fiter() and reduce() Exercise 1: List Comprehension to avoid loop or lambda functions. Find square of each list ‘nums = [76,20,30,40,50] # using for loop new_nums = for num in nums: ‘new_nums.append(num ** 2) new nums # using lambda functions mapilambda x: x “2,qums) list(a) # using list comprehension new_numst = [num ** 2 for num in nums} 97 RR ITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899166, 8790998182RITEC new_numst Exercise 2: List Comprehension to avoid nested loops Print("07, nested loops“) pairs_1 =) for numt in range(0, 2): for num2 in range(6, 8): pairs_1.append((numt, num2)) print(pairs_1) print("02. nested loops using list comprehension ") pairs_2 = [(num1, num2) for numt in range(0, 2)for num2 in range(6, 8)) print(pairs_2) Exercise 3: List Comprehension with conditions Wlist comprehension with if statement print(Inum ** 2 for num in range(10) if num % 2 print("positive and negative numbers”) pos_neg = [{num: -num for num in range(10))) print(pos_neg) pant(type(pos_neg)) Refer: hitps:/]docs.pvthon.ora/3/tutorial/datastructures. htm! SB RRITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899166, 8790998182RITEC 412. Generators 1.Generator functions allow you to declare a function that behaves like an iterator, ie. it can be used in a for loop 2.list comprehensions and generator expressions look very similar in their syntax, except for the use of parentheses () in generator expressions and brackets {Jin fist ‘comprehensions 3.A list comprehension produces a list as output, a generator produces a generator object. in°2 for n in range(1000)] # List comprehension 9 = (n°2 for n in range(1000)) # Generator expression print type())) # type "ist print(type(g)) # import sys Print(sys.getsizeot()) #9032 Print(sys.getsizeot(g)) # 80 Print(ia)) #8 Print(g[4l) # TypeError: ‘generator’ object has no attribute ‘__getitem_* for vin: print) for ving print(v) 4.Exercise 2: work with list and generator # List of strings | sta = (Ram Reddy, ‘Rabson, ‘Robert’, Nancy] # List comprehension std) = {member for member in std if len(member) > 5] | 59 RR ITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374999166, “""9..3182prinitype(stdt)) print(std1) # Generator expression ‘std2 = (member for member ia sid if ten{member) > 5) print(type(std2)) | ist(std2) 5.Read items of generator object using next or for loop. | result = (aur for num in range(t0)) | printt™"** 01 First 5 values of generator **** ") printinent(result)) print(next(resuit)) | printinext(resutt}} } print(next(result)) | print(next(result)) print("*"~* 02 Print the rest of the values using for Loop *"*") for value in result: printivalue) 6.create a generator function with a similar mechanism as the generator expression ¥ Greate a list of stings ~~] kancy)} # Define generator function get_lengths std = [Ram Reddy’, Rabson’, ‘Robert, def get_fengthstinput list): “Generator function that yields the RR ITEC #209, Nitagiri Block, Adithya Enclave, Ameerpet @8374299166, 8790998182__RITEC [meres Sings in inputs # Yield the length of a string for person in input fist: yield len(person) # Print the values generated by got_lenathe() for value in get_lengths(std): print(value) 61 RR ITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @_o/ 4849166, 8790998182RITEC 62 RRITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899166, 8790998182RITEC ‘HHH Level 03 of 08: Python Packages for Data Science ### 13, NumPy package 1. Data Science needs, Mathematical operations over collections of values in utmost no time. It is not possible with regular list. 2. Exercise 1: Calculate BMI using List. Is it possible? © Pye. Colelote BML Using Uist height = (1.72, 1.58, 1.51, 1.73, 1.92] weight = (65.9, 58.8, 68.8, 83.4, 68.7) BMT=woight/height °72 Dconsole # > Py8, 1 Calelate BM Using Uist py [CAUsers\KiAppData\LocalProgrars\Python\Python36 python exe) Traceback (nost recent call 1 File "c:\Users\Hi\uorkspace Bii=weight/height “2 [typetrror: unsupported operand type(s) for ** on peu(}: ‘list’ and TEC PRODECT PyDavisre\Py8 1 Calcul: 63 RR ITEC #209, Nilagiri Block, Adithya Enclave, ..n1eerpet @837489916, 8790998182RITEC 3. Exercise : Calculate BMI using NumPy Array. Is it possible? fe -Pye_2.Calculate,OMILUsing. NumPy Array asi, 1.73, 1.927 weight = [65-9, 58.8, 66.8, 53.4, 68.7] impor sumey . ‘p_haight = numpy-srray height) apiueighe = numpy sreayweigh®) int{eypecap_heigne) | ine Cypetoncweign3 prine (eRr[2]) print (BMx[2:3]) Print (BMx[BMr>29}) 4, NumPy main object is the homogeneous multidimensional array. !tis a table of elements (usually numbers), all of the same type, indexed by a tuple of positive integers. 5. NumPy stands for Numerical Python 6. NumPy array contains only one data type elements whoroas Py elements of different data types Fr import numpy 95 np 2 2 = [2,2,3.9] 5 n= np.array([2,2,3.9]) Js prine(a) = o> Es 5 prine(a) = so: i stay contains 7. NumPy is a powerful N-dimensional array object “import numpy as np a = np.arange(6) b = np.arange(12).reshape(4,3) nge(24).reshape(2,3,4) arange(24)-reshape(2,3,2,2) print(a) print(b) print(e) print(a) GEUnn See vauaune RR ITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899166, 8790998182RITEC f 8. NumPy array allows Calculations over entire elements, Easy, Fast and occupies less fl memory f import ays import numpy as np b = [1,2,3,4,5,6,7,8,9,28,11,12) In = np.array((1,2,3,4,5,6,7,8,9,19,11,12]) lprint(sys.getsizeor(1})’ rint(sys.getsizeof(n)) = 9. Exercise 3: Working with 2D NumPy Arrays. FE -Pya 5working. wih ZONumay Aray = to 2daray « mumpycoreay({3.73, 158, 1.81, 1.73, 1.83], (68.9, 50.8, 66.8, 83.8, 68.719) print(type(np_2e_array)) print(np_24_areay. shape) prine(np_24_orreyte}) - brine(np_2d_array0}{2} ° : print{np=2d-array(:, 0:21) : : peine(np_ad-arrayliys]) =~ fore al 10. Exercise 4: Working with basic statistics. B Py8.4 Working With Statistics import numpy x= [1 4, 8, 18, 12) print (numpy.mean(x)) print(nunpy median(x)) print{numpy.std(x)), 11. https:/www maths! it Auten 12, httos:/Awww.mathsisfun.com/median .ntmt 13, https. d-deviation htm! “Standard Deviation is a “standard” way of knowing what is normal, what is extra-large and what is extra small” 14, Exercise 5: Working with random numbers with out seed # generate 5 random integers between 10(inclusive) and 50(Exclusive) print(np.random randint(10,50,5)) # run again same comman ' and observe is it giving same result? ee 65 RR ITEC #209, Nilagiri Block, Ac..uya enclave, Ameerpet @8374899 166, 8790998182print(nprandom.randint(10,60,6) 15. Exercise 6: Working with random numbers with seed # How to reproduce same random numbers in multiple runs? Let us try # generate 5 random integers between 10(Inclusive) and 50(Exclusive) np.random.seed(100) print(np.random.randint(10,60,5)) #run again same command and observe is it giving same result? np.random.seed(100) print(np.random.randint(10,50,5)) 16. Exercise 6: working with linear algebra # Create a matrix with Ofinclusive) to 12(Exclusive) integers a = np.arange(12).reshape(3,-1) #-1 means ‘whatever is needed” print(a) # Create a matrix with 12 random numbers b = np.array(np.random.rand(12)).reshape(3,-1) #1 means "whatever is needed” _ | print # Add these two matrices print(atb) 17. hitosi/Idocs scipy ora/doc/numpyiuser/quickstart him 66 RRITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899166, 8790998182RITEC 14, Pandas (powerful Python data analysis toolkit) 4141 Introduction 1. Pandas is an open source, BSD-licensed library providing high-performance, easy-to- use data structures and data analysis tools for the Python programming language. 2.Using pandas we can replicate any SQL query 1.Execute below 4 queries and observe the results #01. load csv as dataframe import pandas as pd df = pd.read_csv("pandas_sales.csv",index_col = 'month’) of | #02. Read dataframe # Method 1: by inciex # [columnname]{rowname] dffeggs May] | dit'spam’] # Method 2: by column attribute and row index df.salt{o) . f.salt{0:3] # slice df.salt{{0,3,5]] # fancy # Method 3: by using loc | attoctuan'‘eags 4f.Jocf'Jan'May’,'eggs':'spam’] # slice df loc{{'Jan', May’) fegas''spam]] # fancy # Method 4: bu using iloc dfilocft,1} j | Aailoot:3,1:3|# by sce i 67 RR ITEC #209, N;..gi¢i biock, Adithya Enclave, Ameerpet @8374899166, 8790998182Af.iloc{f1,3]10.2]] # by fancy #03. Create dataframe from datiframe [{ I] df_new = dlfeggs'’spam']] df_new #04. Create series(1 column + series = dff'eggs'] series rR 68 RR ITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899166, 8790998182RITEC 14.3. Filtering Dataframe 4 Execute below 41 queries and observe the results, PyBl_5,Ftering_DataFrame import pandas as pd 2 df = pd.read_csv( ‘pandas_sates.csv', index_col- 3 print("**+r7 "el. observe simple dctafran print(df); — print(""*""" 92. Filter date ~ 5 print(df.salt > 68) =<" © print(df[d#.salt > 66]) 7 print("*7*** 23, Combining a 9 2 Print (AF[(@F-salt >= 50) & Tateggs < 200))) = 520 fonaitions print(""¥+** a4, Dataframes with zeros and Nos af2 = dF .copy() f) = es Te, @, 50, 68, 7, 6) = oy print (df? éf2[ "coke’] print (df2) peint("r7*>" 25, Setect colums with all n print (df2.locf:, df2.a11()}) print( Setzer columns witn any ren: Gf3 = df2.copy() aF3[ ‘mite"] = [2, 0,0,0,0,0) > print (df3) print (dF. 1ocf:, 4#3.any()]) print(""*sr¥ 97, Select columns with any Matis print (df-Loc{:, df-isnul()-any()]) print("t°"+" 28, Setect columns WAthout Ha 4f-loc{:, df.notnull().a11()]) peint ("277 2. Drop rows with any Nats “7*==") print tdf3.dropna¢hows “ony"))) print("=*77" 19, Filtering ¢ eoluan based on another 7772+") print (df eggs[df.salt > 55) peint("***"* 11, Modifying @ colum bazed on another *7****) “@F.eggs[df.salt > 55] t= 5 32 print (df) ———————— 6 RRI- ix #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899166, 8790998182RITEC 144 Transforming Dataframe 1.Execute below 8 queries and observe the results 1B Py31_6 Transforming. Dataframe 1 import pandas as pd if = pd.read_csv( ‘pondas_sates.csv', index_col='month') + at. observe simple datefrane =7**=") print("7" print (df) print (“rest 92. Datorame vectorized methods *7>*>*) print (d¥.floordiv(12)) # © unit print("*e"*" 03. humPy vectorized functions *=7°~ inport numpy 2s np print (np.floor_divide(df, 12)) = conver: to print("****" 04 - 1. Plain Python functions def dozens(n): return n//12 peint (df. apply (dozens)) print("***"* @4 - 2. Plain Python functions * print (df-apply(lambda n: n//12)) x print 25. Storing @ transformation +777") 7 af 'cczens_of_eggs'] = df.eggs.floordiv(12) 8 print (dF) 19 print("*""** 66. The DataFrane index ***"*") 18 print (4F. index) 21 print("*"""" 97 - 4. working sith string votues “*-*=") 22 df index = df-index.ste.upper() 23 print (df) 24 print("*¥**" 07 - 2. Working with string values “*=**") 25 df-Andex = df-index.nap(str-lower) 26 print(df) re 27 print("*r*=* 8. Defining columns using other eSeusnzon<+") 28 af{ ‘salty eggs] = df.salt + df.dozens_of eggs 28 printer) 70 RR ITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899166, 8790998182RITEC 44.8 Advanced indexing 1.1n pandas Data Structures ,Key building b’ocks are a. Indexes: Sequence of labels b. Series: 1D array with Index (index + 1 column) c. DataFrames: 2D array with Series as columns 2.Indexes a. Immutable (Like dictionary keys) b. Homogenous in data type (Like NumPy arrays) 3.Exercise 1: create index manually { import pandas as pd | | prices = | | typetorices) 10,12,13,11,9] # 01. Creating a series shares = pd.Series(prices) | shares type(shares) # 02. Creating an index days = [Mon’, Tue’, Wed’ Thur’ at] shares = pd.Series(data = prices,index = days) shares # 03. Read indexes shares index 7" RR ITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899166, 8790998182| shares index[i] shares.index{:3] shares.index{:-3] # from right side 3 items will be removed shares index{-2:] #04, Index names shares.index.name shares.index.name = ‘weekday’ shares index name shares # 05.Indexs are immutable* shares. index(2] = Wednesday’ #TypeError: Index does not support mutable operations shares index{:2] =[Monday., Tuesday'}#TypeError: Index does not support mutable operations # 06.Modifing all index entries shares.index = [Monday Tuesday’, Wednesday,Thursday\ Friday] shares 4,Exercise 2: Create index from file import import pandas as pd df = pd.read_csv("pandas_sales.csv",jndex_col of 72 RRITEC #209, giti Block, Adithya Enclave, Ameerpet @8374999166, 8790998182. RITEC 5.Exercise 3: Hierarchical indexing #01. load data import pandas as pd df = pd.read_csv("pandas_sales_hierarchical_indexing.csv") of # 02. Create index using two columns (similar to composite key in RDBMS) df = df-set_index({'state’ month’), of # 03. Read index of dataframe df.index df.index.name df.index.names #04, Sort indexes af = df.sort_index() #05. Reading # 05-01. Using index method dflocfCA\1] di.loo{(‘CA’,1),'salt] # 05-02. Using slice df locf CA] di.loof CANNY] 73 RRITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899166, 8790998182# 05-03. Fancy dfiloc{((CA, TX df.loc{(CA',TX].1),"eg9s)] dHloc((CA' (1,2). 14.6 Stack and unstack 1.Recall pivot # Gf. import required modules and load data import pandas as pd at = pd.read_csv("pandas_users.csv") of # 02. weekday and city wise, Number of visitors. visitors_pivot = dt pivot(index = weekday’ columns = ‘city’ value visitors_pivot # 03. weekday and city wise, Number of visitors signups_pivot = di pivot(index = weekday,columns=city, values, signups_pivot # 04.weekday and city wise, number of visitors and signups visitors_signups_pivot = df pivot(index = ‘weekday,columns=city’) visitors_signups_pivot 2.Stack is useful to work with multilevel index a 74 RR ITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899166, 8790998182I ‘#01. import req df = pd.read_csv("pandas_users.csv") of import pandas as pd | #02. set multi level index | af = df.set_ndex(('city, weekday) of #03. by week day | byweekday = df.unstack(level = weekday’) byweekday | 04, revert it | Gf = byweekday.stack(level = ‘weekday’) | on 3.Recall melt and pivot_table from cleaning data chapter a 75 RR ITEC #209, Nilagiri Block, Adithya Enclave, Ameerpet @8374899166, 8790998182