Python Book 01

APythonBook:BeginningPython,Advanced
Python,andPythonExercises
Author:
DaveKuhlman
Address:
dkuhlman@rexx.comx
http://www.rexx.com/~dkuhlman
Page1
revision
1.0a
date
March31,2009
Copyright
Copyright(c)2009DaveKuhlman.AllRightsReserved.Thisdocumentis
subjecttotheprovisionsoftheOpenSourceMITLicense
http://www.opensource.org/licenses/mitlicense.php.
Abstract
ThisdocumentisaselflearningdocumentforacourseinPythonprogramming.
Thiscoursecontains(1)apartforbeginners,(2)adiscussionofseveral
advancedtopicsthatareofinteresttoPythonprogrammers,and(3)aPython
workbookwithlotsofexercises.
Page2
TableofContents
1Part1BeginningPython........................................................................................8
1.1IntroductionPython101BeginningPython...............................................8
1.1.1ImportantFeaturesofPython....................................................................8
1.1.2WheretoGoForAdditionalhelp..............................................................9
1.2InteractivePython..............................................................................................9
1.3Lexicalmatters................................................................................................10
1.3.1Lines.........................................................................................................10
1.3.2NamesandTokens...................................................................................10
1.3.3BlocksandIndentation............................................................................11
1.3.4DocStrings..............................................................................................11
1.3.5Operators..................................................................................................11
1.3.6AlsoSee...................................................................................................12
1.3.7CodeEvaluation.......................................................................................13
1.4BuiltinDataTypes.........................................................................................13
1.4.1Strings......................................................................................................13
1.4.1.1Whatstringsare................................................................................13
1.4.1.2Whentousestrings..........................................................................13
1.4.1.3Howtousestrings............................................................................14
1.4.2SequencesListsandTuples..................................................................16
1.4.2.1Whatsequencesare..........................................................................16
1.4.2.2Whentousesequences.....................................................................17
1.4.2.3Howtousesequences......................................................................17
1.4.3Dictionaries..............................................................................................19
1.4.3.1Whatdictionariesare........................................................................19
1.4.3.2Whentousedictionaries..................................................................20
1.4.3.3Howtousedictionaries....................................................................20
1.4.4Files..........................................................................................................23
1.4.4.1Whatfilesare...................................................................................23
1.4.4.2Whentousefiles..............................................................................23
1.4.4.3Howtousefiles................................................................................23
1.4.4.4ReadingTextFiles...........................................................................23
1.5SimpleStatements...........................................................................................25
1.5.1printstatement..........................................................................................25
1.5.2Assignmentstatement..............................................................................26
1.5.3importstatement......................................................................................27
Page3
1.5.4assertstatement........................................................................................28
1.5.5globalstatement.......................................................................................29
1.6CompoundstatmentsControlStructures.....................................................30
1.6.1if:statement..............................................................................................30
1.6.2for:statement...........................................................................................31
1.6.2.1Thefor:statementandunpacking....................................................33
1.6.3while:statement.......................................................................................34
1.6.4try:except:andraiseExceptions...........................................................35
1.7Organization....................................................................................................37
1.7.1Functions..................................................................................................37
1.7.1.1Abasicfunction...............................................................................37
1.7.1.2Afunctionwithdefaultarguments...................................................37
1.7.1.3Argumentlistsandkeywordargumentlists.....................................37
1.7.1.4Callingafunctionwithkeywordarguments....................................38
1.7.2Classesandinstances...............................................................................39
1.7.2.1Abasicclass.....................................................................................39
1.7.2.2Inheritance........................................................................................40
1.7.2.3Classdata.........................................................................................41
1.7.2.4Staticmethodsandclassmethods....................................................41
1.7.2.5Properties..........................................................................................43
1.7.3Modules...................................................................................................44
1.7.4Packages...................................................................................................46
1.8AcknowledgementsandThanks......................................................................47
1.9SeeAlso...........................................................................................................48
2Part2AdvancedPython......................................................................................49
2.1IntroductionPython201(Slightly)AdvancedPythonTopics.................49
2.2RegularExpressions........................................................................................49
2.2.1Definingregularexpressions...................................................................49
2.2.2Compilingregularexpressions................................................................50
2.2.3Usingregularexpressions........................................................................50
2.2.4Usingmatchobjectstoextractavalue....................................................51
2.2.5Extractingmultipleitems.........................................................................52
2.2.6Replacingmultipleitems.........................................................................53
2.3IteratorObjects................................................................................................55
2.3.1ExampleAgeneratorfunction...............................................................57
2.3.2ExampleAclasscontainingageneratormethod...................................58
2.3.3ExampleAniteratorclass......................................................................59
2.3.4ExampleAniteratorclassthatusesyield..............................................61
2.3.5ExampleAlistcomprehension..............................................................62
2.3.6ExampleAgeneratorexpression...........................................................62
2.4UnitTests.........................................................................................................63
2.4.1Definingunittests....................................................................................63
2.4.1.1Createatestclass.............................................................................63
2.5ExtendingandembeddingPython...................................................................65
2.5.1Introductionandconcepts........................................................................65
2.5.2Extensionmodules...................................................................................66
Page4
2.5.3SWIG.......................................................................................................68
2.5.4Pyrex........................................................................................................71
2.5.5SWIGvs.Pyrex.......................................................................................75
2.5.6Cython......................................................................................................76
2.5.7Extensiontypes........................................................................................77
2.5.8Extensionclasses.....................................................................................77
2.6Parsing.............................................................................................................78
2.6.1Specialpurposeparsers............................................................................78
2.6.2Writingarecursivedescentparserbyhand.............................................78
2.6.3Creatingalexer/tokenizerwithPlex........................................................85
2.6.4Asurveyofexistingtools........................................................................94
2.6.5CreatingaparserwithPLY......................................................................94
2.6.6Creatingaparserwithpyparsing...........................................................100
2.6.6.1Parsingcommadelimitedlines......................................................100
2.6.6.2Parsingfunctors..............................................................................101
2.6.6.3Parsingnames,phonenumbers,etc...............................................102
2.6.6.4Amorecomplexexample..............................................................103
2.7GUIApplications...........................................................................................105
2.7.1Introduction............................................................................................105
2.7.2PyGtk.....................................................................................................105
2.7.2.1Asimplemessagedialogbox.........................................................105
2.7.2.2Asimpletextinputdialogbox.......................................................107
2.7.2.3Afileselectiondialogbox.............................................................109
2.7.3EasyGUI.................................................................................................111
2.7.3.1AsimpleEasyGUIexample...........................................................112
2.7.3.2AnEasyGUIfileopendialogexample..........................................112
2.8GuidanceonPackagesandModules.............................................................112
2.8.1Introduction............................................................................................112
2.8.2ImplementingPackages.........................................................................112
2.8.3UsingPackages......................................................................................113
2.8.4DistributingandInstallingPackages.....................................................113
2.9EndMatter.....................................................................................................114
2.9.1AcknowledgementsandThanks............................................................115
2.9.2SeeAlso.................................................................................................115
3Part3PythonWorkbook...................................................................................116
3.1Introduction...................................................................................................116
3.2LexicalStructures..........................................................................................116
3.2.1Variablesandnames..............................................................................116
3.2.2Linestructure.........................................................................................118
3.2.3Indentationandprogramstructure.........................................................119
3.3ExecutionModel............................................................................................120
3.4BuiltinDataTypes.......................................................................................121
3.4.1Numbers.................................................................................................121
3.4.1.1Literalrepresentationsofnumbers.................................................121
3.4.1.2Operatorsfornumbers....................................................................123
3.4.1.3Methodsonnumbers......................................................................125
Page5
3.4.2Lists........................................................................................................125
3.4.2.1Literalrepresentationoflists..........................................................126
3.4.2.2Operatorsonlists............................................................................128
3.4.2.3Methodsonlists.............................................................................128
3.4.2.4Listcomprehensions.......................................................................129
3.4.3Strings....................................................................................................131
3.4.3.1Characters.......................................................................................132
3.4.3.2Operatorsonstrings.......................................................................133
3.4.3.3Methodsonstrings.........................................................................134
3.4.3.4Rawstrings.....................................................................................136
3.4.3.5Unicodestrings...............................................................................137
3.4.4Dictionaries............................................................................................138
3.4.4.1Literalrepresentationofdictionaries.............................................138
3.4.4.2Operatorsondictionaries...............................................................139
3.4.4.3Methodsondictionaries.................................................................140
3.4.5Files........................................................................................................143
3.4.6Afewmiscellaneousdatatypes.............................................................145
3.4.6.1None...............................................................................................145
3.4.6.2ThebooleansTrueandFalse..........................................................145
3.5Statements......................................................................................................145
3.5.1Assignmentstatement............................................................................146
3.5.2printstatement........................................................................................147
3.5.3if:statementexercises............................................................................148
3.5.4for:statementexercises..........................................................................149
3.5.5while:statementexercises......................................................................152
3.5.6breakandcontinuestatements...............................................................153
3.5.7Exceptionsandthetry:except:andraisestatements..............................154
3.6Functions.......................................................................................................156
3.6.1Optionalargumentsanddefaultvalues..................................................157
3.6.2Passingfunctionsasarguments.............................................................159
3.6.3Extraargsandkeywordargs..................................................................160
3.6.3.1Orderofarguments(positional,extra,andkeywordargs).............162
3.6.4Functionsandducktypingandpolymorphism.....................................162
3.6.5Recursivefunctions................................................................................163
3.6.6Generatorsanditerators.........................................................................165
3.7Objectorientedprogrammingandclasses....................................................168
3.7.1Theconstructor......................................................................................168
3.7.2InheritanceImplementingasubclass.................................................170
3.7.3Classesandpolymorphism....................................................................171
3.7.4Recursivecallstomethods.....................................................................172
3.7.5Classvariables,classmethods,andstaticmethods...............................174
3.7.5.1Decoratorsforclassmethodandstaticmethod................................176
3.8AdditionalandAdvancedTopics..................................................................178
3.8.1Decoratorsandhowtoimplementthem................................................178
3.8.1.1Decoratorswitharguments............................................................179
3.8.1.2Stackeddecorators.........................................................................180
Page6
3.8.1.3Morehelpwithdecorators.............................................................182
3.8.2Iterables..................................................................................................182
3.8.2.1AfewpreliminariesonIterables....................................................182
3.9ApplicationsandRecipies.............................................................................183
3.9.1XML.......................................................................................................184
3.9.2Relationaldatabaseaccess.....................................................................191
3.9.3CSVcommaseparatedvaluefiles......................................................196
3.9.4YAMLandPyYAML............................................................................197
3.9.5Json........................................................................................................198
Page7
[tableofcontents]
1Part1BeginningPython
1.1IntroductionPython101BeginningPython
Pythonisahighlevelgeneralpurposeprogramminglanguage:
Becausecodeisautomaticallycompiledtobytecodeandexecuted,Pythonis
suitableforuseasascriptinglanguage,Webapplicationimplementation
language,etc.
BecausePythoncanbeextendedinCandC++,Pythoncanprovidethespeed
neededforevencomputeintensivetasks.
Becauseofitsstrongstructuringconstructs(nestedcodeblocks,functions,
classes,modules,andpackages)anditsconsistentuseofobjectsandobject
orientedprogramming,Pythonenablesustowriteclear,logicalapplications
forsmallandlargetasks.
1.1.1ImportantFeaturesofPython
Builtinhighleveldatatypes:strings,lists,dictionaries,etc.
Theusualcontrolstructures:if,ifelse,ifelifelse,while,plusapowerful
collectioniterator(for).
Multiplelevelsoforganizationalstructure:functions,classes,modules,and
packages.Theseassistinorganizingcode.Anexcellentandlargeexampleis
thePythonstandardlibrary.
CompileontheflytobytecodeSourcecodeiscompiledtobytecode
withoutaseparatecompilestep.Sourcecodemodulescanalsobe"pre
compiled"tobytecodefiles.
ObjectorientedPythonprovidesaconsistentwaytouseobjects:everything
isanobject.And,inPythonitiseasytoimplementnewobjecttypes(called
classesinobjectorientedprogramming).
ExtensionsinCandC++Extensionmodulesandextensiontypescanbe
writtenbyhand.Therearealsotoolsthathelpwiththis,forexample,SWIG,
sip,Pyrex.
JythonisaversionofPythonthat"playswellwith"Java.See:TheJython
Projecthttp://www.jython.org/Project/.
Somethingsyouwillneedtoknow:
Page8
Pythonusesindentationtoshowblockstructure.Indentoneleveltoshowthe
beginningofablock.Outdentoneleveltoshowtheendofablock.Asan
example,thefollowingCstylecode:
if(x)
{
if(y)
{
f1()
}
f2()
}
inPythonwouldbe:
ifx:
ify:
f1()
f2()
And,theconventionistousefourspaces(andnohardtabs)foreachlevelof
indentation.Actually,it'smorethanaconvention;it'spracticallyarequirement.
Followingthat"convention"willmakeitsomucheasiertomergeyourPythoncode
withcodefromothersources.
1.1.2WheretoGoForAdditionalhelp
ThestandardPythondocumentationsetItcontainsatutorial,alanguage
reference,thestandardlibraryreference,anddocumentsonextendingPython
inC/C++.Youcanfindithere:http://www.python.org/doc/.
OtherPythontutorialsSeeespecially:
Beginner'sGuidetoPythonhttp://wiki.python.org/moin/BeginnersGuide
OtherPythonresourcesSeeespecially:
Pythondocumentationhttp://www.python.org/doc/
ThePythonhomeWebsitehttp://www.python.org/
ThewholePythonFAQhttp://www.python.org/doc/FAQ.html
1.2InteractivePython
IfyouexecutePythonfromthecommandlinewithnoscript(noarguments),Python
givesyouaninteractiveprompt.ThisisanexcellentfacilityforlearningPythonand
fortryingsmallsnippetsofcode.Manyoftheexamplesthatfollowweredeveloped
usingthePythoninteractiveprompt.
StartthePythoninteractiveinterpreterbytypingpythonwithnoargumentsatthe
commandline.Forexample:
$python
Python2.6.1(r261:67515,Jan112009,15:19:23)
[GCC4.3.2]onlinux2
Type"help","copyright","credits"or"license"formore
information.
>>>print'hello'
hello
Page9
>>>
YoumayalsowanttoconsiderusingIDLE.IDLEisagraphicalintegrated
developmentenvironmentforPython;itcontainsaPythonshell.ItislikelythatIdle
wasinstalledforyouwhenyouinstalledPython.Youwillfindascripttostartup
IDLEintheTools/scriptsdirectoryofyourPythondistribution.IDLErequires
Tkinter.
Inaddition,therearetoolsthatwillgiveyouamorepowerfulandfancyPython
interactiveinterpreter.OneexampleisIPython,whichisavailableat
http://ipython.scipy.org/.
1.3Lexicalmatters
1.3.1Lines
Pythondoeswhatyouwantittodomostofthetimesothatyouonlyhaveto
addextracharacterssomeofthetime.
Statementseparatorisasemicolon,butisonlyneededwhenthereismore
thanonestatementonaline.
ContinuationlinesUseabackslashattheendoftheline.But,notethatan
openingbracket(orparenthesis)makethebackslashunnecessary.
CommentsEverythingafter"#"onalineisignored.Noblockcomments,
butdocstringsareacommentinquotesatthebeginningofamodule,class,
methodorfunction.Also,editorswithsupportforPythonwillcommentouta
selectedblockofcode,usuallywith"##".
1.3.2NamesandTokens
Allowedcharactersinaname:azAZ09underscore,andmustbeginwitha
letterorunderscore.
Namesandidentifiersarecasesensitive.
Identifierscanbeofunlimitedlength.
Specialnames,customizing,etc.Usuallybeginandendindouble
underscores.
SpecialnameclassesSingleanddoubleunderscores.
LeadingdoubleunderscoresNamemanglingformethodnames.
LeadingsingleunderscoreSuggestsa"private"methodnameinaclass.
Notimportedby"frommoduleimport*".
TrailingsingleunderscoreSometimesusedtoavoidaconflictwitha
keyword,forexample,class_.
NamingconventionsNotrigid,buthereisonesetofrecommendations:
Modulesandpackagesalllowercase.
GlobalsandconstantsUppercase.
ClassnamesBumpycapswithinitialupper.
MethodandfunctionnamesAlllowercasewithwordsseparatedby
underscores.
Page10
LocalvariablesLowercase(possiblywithunderscorebetweenwords)
orbumpycapswithinitialloweroryourchoice.
Names/variablesinPythondonothaveatype.Valueshavetypes.
1.3.3BlocksandIndentation
Pythonrepresentsblockstructureandnestedblockstructurewithindentation,not
withbeginandendbrackets.
TheemptyblockUsethepassnoopstatement.
Benefitsoftheuseofindentationtoindicatestructure:
Reducestheneedforacodingstandard.Onlyneedtospecifythatindentation
is4spacesandnohardtabs.
Reducesinconsistency.Codefromdifferentsourcesfollowthesame
indentationstyle.Ithasto.
Reduceswork.Onlyneedtogettheindentationcorrect,notbothindentation
andbrackets.
Reducesclutter.Eliminatesallthecurlybrackets.
Ifitlookscorrect,itiscorrect.Indentationcannotfoolthereader.
EditorconsiderationsThestandardforindentingPythoncodeis4spaces(nohard
tabs)foreachindentationlevel.Youwillneedatexteditorthathelpsyourespectthat.
1.3.4DocStrings
Docstringsarelikecomments,buttheyarecarriedwithexecutingcode.Docstrings
canbeviewedwithseveraltools,e.g.help(),obj.__doc__,and,inIPython,a
questionmark(?)afteranamewillproducehelp.
Adocstringisaquotedstringatthebeginningofamodule,function,class,or
method.
Wecanusetriplequotingtocreatedocstringsthatspanmultiplelines.
Therearealsotoolsthatextractandformatdocstrings,forexample:
pydocDocumentationgeneratorandonlinehelpsystem.Seepydoc
http://docs.python.org/lib/modulepydoc.html.
epydocAutomaticAPIDocumentationGenerationforPython.SeeEpydoc
http://epydoc.sourceforge.net/index.html
SphinxSphinxisapowerfultoolforgeneratingPythondocumentation.
See:SphinxPythonDocumentationGeneratorhttp://sphinx.pocoo.org/.
1.3.5Operators
See:http://docs.python.org/ref/operators.html.Pythondefinesthefollowing
operators:
+***///%
<<>>&|^~
<><=>===!=<>
Page11
Thecomparisonoperators<>and!=arealternatespellingsofthesame
operator.!=isthepreferredspelling;<>isobsolescent.
Logicaloperators:
andorisnotin
Therearealso(1)thedotoperator,(2)thesubscriptoperator[],andthe
function/methodcalloperator().
Forinformationontheprecedencesofoperators,seeSummaryofoperators
http://docs.python.org/ref/summary.html,whichisreproducedbelow.
ThefollowingtablesummarizestheoperatorprecedencesinPython,from
lowestprecedence(leastbinding)tohighestprecedence(mostbinding).
Operatorsonthesamelinehavethesameprecedence.Unlessthesyntaxis
explicitlygiven,operatorsarebinary.Operatorsonthesamelinegroupleftto
right(exceptforcomparisons,includingtests,whichallhavethesame
precedenceandchainfromlefttorightseesection5.9and
exponentiation,whichgroupsfromrighttoleft):
OperatorDescription
==========================================
lambdaLambdaexpression
orBooleanOR
andBooleanAND
notxBooleanNOT
in,notinMembershiptests
is,isnotIdentitytests
<,<=,>,>=,<>,!=,==Comparisons
|BitwiseOR
^BitwiseXOR
&BitwiseAND
<<,>>Shifts
+,Additionandsubtraction
*,/,%Multiplication,division,
remainder
+x,xPositive,negative
~xBitwisenot
**Exponentiation
x.attributeAttributereference
x[index]Subscription
x[index:index]Slicing
f(arguments...)Functioncall
(expressions...)Bindingortupledisplay
[expressions...]Listdisplay
{key:datum...}Dictionarydisplay
èxpressions...`Stringconversion
Notethatmostoperatorsresultincallstomethodswithspecialnames,for
example__add__,__sub__,__mul__,etc.SeeSpecialmethodnames
http://docs.python.org/ref/specialnames.html
Later,wewillseehowtheseoperatorscanbeemulatedinclassesthatyou
defineyourself,throughtheuseofthesespecialnames.
1.3.6AlsoSee
FormoreonlexicalmattersandPythonstyles,see:
Page12
CodeLikeaPythonista:IdiomaticPython
http://python.net/~goodger/projects/pycon/2007/idiomatic/handout.html.
StyleGuideforPythonCodehttp://www.python.org/dev/peps/pep0008/
1.3.7CodeEvaluation
UnderstandingthePythonexecutionmodelHowPythonevaluatesandexecutes
yourcode.Pythonevaluatesascriptormodulefromthetoptothebottom,binding
values(objects)tonamesasitproceeds.
EvaluatingexpressionsExpressionsareevaluatedinkeepingwiththerules
describedforoperators,above.
Creatingnames/variablesBindingThefollowingallcreatenames(variables)and
bindvalues(objects)tothem:(1)assignment,(2)functiondefinition,(3)class
definition,(4)functionandmethodcall,(5)importingamodule,...
FirstclassobjectsAlmostallobjectsinPythonarefirstclass.Definition:Anobject
isfirstclassif:(1)wecanputitinastructuredobject;(2)wecanpassittoafunction;
and(3)wecanreturnitfromafunction.
ReferencesObjects(orreferencestothem)canbeshared.Whatdoesthismean?
Theobject(s)satisfytheidentitytestoperatoris,thatis,obj1 is obj2returns
True.
Thebuiltinfunctionid(obj)returnsthesamevalue,thatis,id(obj1) ==
id(obj2)isTrue.
Theconsequencesformutableobjectsaredifferentfromthoseforimmutable
objects.
Changing(updating)amutableobjectreferencedthroughonevariableor
containeralsochangesthatobjectreferencedthroughothervariablesor
containers,becauseitisthesameobject.
del()Thebuiltinfunctiondel()removesareference,not(necessarily)the
objectitself.
1.4BuiltinDataTypes
1.4.1Strings
1.4.1.1Whatstringsare
InPython,stringsareimmutablesequencesofcharacters.Theyareimmutableinthat
inordertomodifyastring,youmustproduceanewstring.
1.4.1.2Whentousestrings
Anytextinformation.
Page13
1.4.1.3Howtousestrings
Createanewstringfromaconstant:
s1='abce'
s2="xyz"
s3="""A
multiline
string.
"""
Useanyofthestringmethods,forexample:
>>>'Thehappycatranhome.'.upper()
'THEHAPPYCATRANHOME.'
>>>'Thehappycatranhome.'.find('cat')
10
>>>'Thehappycatranhome.'.find('kitten')
1
>>>'Thehappycatranhome.'.replace('cat','dog')
'Thehappydogranhome.'
Type"help(str)"orseehttp://www.python.org/doc/current/lib/stringmethods.htmlfor
moreinformationonstringmethods.
Youcanalsousetheequivalentfunctionsfromthestringmodule.Forexample:
>>>importstring
>>>s1='Thehappycatranhome.'
>>>string.find(s1,'happy')
4
SeestringCommonstringoperations
http://www.python.org/doc/current/lib/modulestring.htmlformoreinformationon
thestringmodule.
Thereisalsoastringformattingoperator:"%".Forexample:
>>>state='California'
>>>'Itneverrainsinsunny%s.'%state
'ItneverrainsinsunnyCalifornia.'
>>>
>>>width=24
>>>height=32
>>>depth=8
>>>print'Theboxis%dby%dby%d.'%(width,height,depth,)
Theboxis24by32by8.
Thingstoknow:
Formatspecifiersconsistofapercentsignfollowedbyflags,length,anda
typecharacter.
Thenumberofformatspecifiersinthetargetstring(totheleftofthe"%"
operator)mustbethesameasthenumberofvaluesontheright.
Whentherearemorethanonevalue(ontheright),theymustbeprovidedina
tuple.
Youcanlearnaboutthevariousconversioncharactersandflagsusedtocontrolstring
Page14
formattinghere:StringFormattingOperations
http://docs.python.org/library/stdtypes.html#stringformattingoperations.
Youcanalsowritestringstoafileandreadthemfromafile.Herearesome
examples.
WritingForexample:
>>>outfile=open('tmp.txt','w')
>>>outfile.write('Thisisline#1\n')
>>>outfile.close()
Notes:
Notetheendoflinecharacterattheendofeachstring.
Theopen()builtinfunctioncreatesafileobject.Ittakesasarguments(1)the
filenameand(2)amode.Commonlyusedmodesare"r"(read),"w"(write),
and"a"(append).
SeeBuiltinFunctions:open()
http://docs.python.org/library/functions.html#openformoreinformationon
openingfiles.SeeBuiltinTypes:FileObjects
http://docs.python.org/library/stdtypes.html#fileobjectsformoreinformation
onhowtousefileobjects.
Readinganentirefileexample:
>>>infile=file('tmp.txt','r')
>>>content=infile.read()
>>>printcontent
Thisisline#1
Thisisline#2
Thisisline#3
>>>infile.close()
Notes:
Alsoconsiderusingsomethinglikecontent.splitlines(),ifyouwanttodivide
contentinlines(splitonnewlinecharacters).
Readingafileonelineatatimeexample:
>>>forlineininfile:
...print'Line:',line
...
Line:Thisisline#1
Line:Thisisline#2
Line:Thisisline#3
>>>infile.close()
Notes:
Page15
Learnmoreaboutthefor:statementinsectionfor:statement.
"infile.readlines()"returnsalistoflinesinthefile.Forlargefilesusethefile
objectitselfor"infile.xreadlines()",bothofwhichareiteratorsforthelinesin
thefile.
InolderversionsofPython,afileobjectisnotitselfaniterator.Inthoseolder
versionsofPython,youmayneedtouseinfile.readlines()orawhileloop
containinginfile.readline()Forexample:
>>>forlineininfile.readlines():
...print'Line:',line
...
Afewadditionalcommentsaboutstrings:
Astringisaspecialkindofsequence.So,youcanindexintothecharactersof
astringandyoucaniterateoverthecharactersinastring.Forexample:
>>>s1='abcd'
>>>s1[1]
'b'
>>>s1[2]
'c'
>>>forchins1:
...printch
...
a
b
c
d
Ifyouneedtodofastorcomplexstringsearches,thereisaregularexpression
moduleinthestandardlibrary.rereRegularexpressionoperations
http://docs.python.org/library/re.html.
Aninterestingfeatureofstringformattingistheabilitytousedictionariesto
supplythevaluesthatareinserted.Hereisanexample:
names={'tree':'sycamore','flower':'poppy','herb':
'arugula'}
print'Thetreeis%(tree)s'%names
print'Thefloweris%(flower)s'%names
print'Theherbis%(herb)s'%names
1.4.2SequencesListsandTuples
1.4.2.1Whatsequencesare
ThereareseveraltypesofsequencesinPython.We'vealreadydiscussedstrings,
whicharesequencesofcharacters.Inthissectionwewilldescribelistsandtuples.
SeeBuiltinTypes:SequenceTypesstr,unicode,list,tuple,buffer,xrange
http://docs.python.org/library/stdtypes.html#sequencetypesstrunicodelisttuple
bufferxrangeformoreinformationonPython'sbuiltinsequencetypes.
Listsaredynamicarrays.Theyarearraysinthesensethatyoucanindexitemsina
Page16
list(forexample"mylist[3]")andyoucanselectsubranges(forexample
"mylist[2:4]").Theyaredynamicinthesensethatyoucanaddandremoveitemsafter
thelistiscreated.
Tuplesarelightweightlists,butdifferfromlistsinthattheyareimmutable.Thatis,
onceatuplehasbeencreated,youcannotmodifyit.Youcan,ofcourse,modifyany
(modifiable)objectsthatthetuplecontains,inotherwordsthatitrefersto.
Capabilitiesoflists:
Appendanitem.
Insertanitem(atthebeginningorintothemiddleofthelist).
Addalistofitemstoanexistinglist.
Capabilitiesoflistsandtuples:
Indexitems,thatisgetanitemoutofalistortuplebasedonthepositionin
thelist(relativetozero,thebeginningofthesequence).
Selectasubsequenceofcontiguousitems(alsoknownasaslice).
Iterateovertheitemsinthelistortuple.
1.4.2.2Whentousesequences
Wheneveryouwanttoprocessacolletionofitems.
Wheneveryouwanttoiterateoveracollectionofitems.
Wheneveryouwanttoindexintoacollectionofitems.
CollectionsNotallcollectionsinPythonareorderedsequences.Hereisa
comparisonofsomedifferenttypesofcollectionsinPythonandtheircharacteristics:
Stringordered,characters,immutable
Tupleordered,heterogeneous,immutable
Listordered,heterogeneous,mutable
Dictionaryunordered,key/valuespairs,mutable
Setunordered,heterogeneous,mutable,uniquevalues
1.4.2.3Howtousesequences
Tocreatealistusesquarebrackets.Examples:
>>>items=[111,222,333]
>>>items
[111,222,333]
Createanewlistorcopyanexistingonewiththelistconstructor:
>>>trees1=list(['oak','pine','sycamore'])
>>>trees1
['oak','pine','sycamore']
>>>trees2=list(trees1)
>>>trees2
['oak','pine','sycamore']
>>>trees1istrees2
False
Tocreateatuple,usecommas,andpossiblyparenthesesaswell:
Page17
>>>a=(11,22,33,)
>>>b='aa','bb'
>>>c=123,
>>>a
(11,22,33)
>>>b
('aa','bb')
>>>c
(123,)
>>>type(c)
<type'tuple'>
Notes:
Tocreateatuplecontainingasingleitem,westillneedthecomma.Example:
>>>print('abc',)
('abc',)
>>>type(('abc',))
<type'tuple'>
Toaddanitemtotheendofalist,useappend():
>>>items.append(444)
>>>items
[111,222,333,444]
Toinsertanitemintoalist,useinsert().Thisexampleinsertsanitematthe
beginningofalist:
>>>items.insert(0,1)
>>>items
[1,111,222,333,444]
Toaddtwoliststogether,creatinganewlist,usethe+operator.Toaddtheitemsin
onelisttoanexistinglist,usetheextend()method.Examples:
>>>a=[11,22,33,]
>>>b=[44,55]
>>>c=a+b
>>>c
[11,22,33,44,55]
>>>a
[11,22,33]
>>>b
[44,55]
>>>a.extend(b)
>>>a
[11,22,33,44,55]
Youcanalsopushitemsontotherightendofalistandpopitemsofftherightendof
alistwithappend()andpop().Thisenablesustousealistasastacklikedata
structure.Example:
>>>items=[111,222,333,444,]
>>>items
[111,222,333,444]
>>>items.append(555)
>>>items
Page18
[111,222,333,444,555]
>>>items.pop()
555
>>>items
[111,222,333,444]
And,youcaniterateovertheitemsinalistortuple(orothercollection,forthat
matter)withthefor:statement:
>>>foriteminitems:
...print'item:',item
...
item:1
item:111
item:222
item:333
item:444
Formoreonthefor:statement,seesectionfor:statement.
1.4.3Dictionaries
1.4.3.1Whatdictionariesare
Adictionaryis:
Anassociativearray.
Amappingfromkeystovalues.
Acontainer(collection)thatholdskeyvaluepairs.
Adictionaryhasthefollowingcapabilities:
Abilitytoiterateoverkeysorvaluesorkeyvaluepairs.
Abilitytoaddkeyvaluepairsdynamically.
Abilitytolookupavaluebykey.
Forhelpondictionaries,type:
>>>helpdict
atPython'sinteractiveprompt,or:
$pydocdict
atthecommandline.
Italsomaybehelpfultousethebuiltindir()function,thentoaskforhelpona
specificmethod.Example:
>>>a={}
>>>dir(a)
['__class__','__cmp__','__contains__','__delattr__',
'__delitem__','__doc__','__eq__','__format__','__ge__',
'__getattribute__','__getitem__','__gt__','__hash__',
'__init__',
'__iter__','__le__','__len__','__lt__','__ne__','__new__',
'__reduce__','__reduce_ex__','__repr__','__setattr__',
'__setitem__','__sizeof__','__str__','__subclasshook__',
Page19
'clear',
'copy','fromkeys','get','has_key','items','iteritems',
'iterkeys','itervalues','keys','pop','popitem','setdefault',
'update','values']
>>>
>>>help(a.keys)
Helponbuiltinfunctionkeys:
keys(...)
D.keys()>listofD'skeys
Moreinformationaboutdictionaryobjectsisavailablehere:Mappingtypesdict
http://docs.python.org/library/stdtypes.html#mappingtypesdict.
1.4.3.2Whentousedictionaries
Whenyouneedlookupbykey.
Whenyouneeda"structured"liteweightobjectoranobjectwithnamed
fields.(But,don'tforgetclasses,whichyouwilllearnaboutlaterinthis
document.)
Whenyouneedtomapanameorlabeltoanykindofobject,evenan
executableonesuchasafunction.
1.4.3.3Howtousedictionaries
Createadictionarywithcurlybracets.Itemsinadictionaryareseparatebycommas.
Useacolonbetweeneachkeyanditsassociatedvalue:
>>>lookup={}
>>>lookup
{}
>>>states={'az':'Arizona','ca':'California'}
>>>states['ca']
'California'
or:
>>>deffruitfunc():
...print"I'mafruit."
>>>defvegetablefunc():
...print"I'mavegetable."
>>>
>>>lookup={'fruit':fruitfunc,'vegetable':vegetablefunc}
>>>lookup
{'vegetable':<functionvegetablefuncat0x4028980c>,
'fruit':<functionfruitfuncat0x4028e614>}
>>>lookup['fruit']()
I'mafruit.
>>>lookup['vegetable']()
I'mavegetable.
or:
>>>lookup=dict((('aa',11),('bb',22),('cc',33)))
>>>lookup
{'aa':11,'cc':33,'bb':22}
Page20
Notethatthekeysinadictionarymustbeimmutable.Therefore,youcanuseanyof
thefollowingaskeys:numbers,strings,tuples.
Testfortheexistenceofakeyinadictionarywiththeinoperator:
>>>if'fruit'inlookup:
...print'containskey"fruit"'
...
containskey"fruit"
or,alternatively,usethe(slightlyoutdated)has_key()method:
>>>iflookup.has_key('fruit'):
...print'containskey"fruit"'
...
containskey"fruit"
Accessthevalueassociatedwithakeyinadictionarywiththeindexingoperator
(squarebrackets):
>>>printlookup['fruit']
<functionfruitfuncat0x4028e614>
Noticethattheabovewillthrowanexceptionifthekeyisnotinthedictionary:
>>>printlookup['salad']
Traceback(mostrecentcalllast):
File"<stdin>",line1,in<module>
KeyError:'salad'
Andso,theget()methodisaneasywaytogetavaluefromadictionarywhile
avoidinganexception.Forexample:
>>>printlookup.get('fruit')
>>>printlookup.get('salad')
None
>>>printlookup.get('salad',fruitfunc)
Adictionaryisaniteratorobjectthatproducesitskeys.So,wecaniterateoverthe
keysinadictionaryasfollows:
>>>forkeyinlookup:
...print'key:%s'%key
...lookup[key]()
...
key:vegetable
I'mavegetable.
key:fruit
I'mafruit.
And,rememberthatyoucansubclassdictionaries.Herearetwoversionsofthesame
example.ThekeywordargumentsinthesecondversionrequirePython2.3orlater:
#
#ThisexampleworkswithPython2.2.
classMyDict_for_python_22(dict):
Page21
def__init__(self,**kw):
forkeyinkw.keys():
self[key]=kw[key]
defshow(self):
print'ShowingexampleforPython2.2...'
forkeyinself.keys():
print'key:%svalue:%s'%(key,self[key])
deftest_for_python_22():
d=MyDict_for_python_22(one=11,two=22,three=33)
d.show()
test_for_python_22()
AversionfornewerversionsofPython:
#
#ThisexampleworkswithPython2.3ornewerversionsofPython.
#Keywordsupport,whensubclassingdictionaries,seemstohave
#beenenhancedinPython2.3.
classMyDict(dict):
defshow(self):
print'ShowingexampleforPython2.3ornewer.'
forkeyinself.keys():
print'key:%svalue:%s'%(key,self[key])
deftest():
d=MyDict(one=11,two=22,three=33)
d.show()
test()
Runningthisexampleproduces:
ShowingexampleforPython2.2...
key:onevalue:11
key:threevalue:33
key:twovalue:22
ShowingexampleforPython2.3ornewer.
key:threevalue:33
key:twovalue:22
key:onevalue:11
Afewcommentsaboutthisexample:
LearnmoreaboutclassesandhowtoimplementtheminsectionClassesand
instances.
TheclassMyDictdoesnotdefineaconstructor(__init__).Thisenablesusto
reusethecontructorfromsuperclassdictandanyofitsforms.Type"help
dict"atthePythoninteractiveprompttolearnaboutthevariouswaystocall
thedictconstructor.
Theshowmethodisthespecializationaddedtooursubclass.
Inoursubclass,wecanrefertoanymethodsinthesuperclass(dict).For
example:self.keys().
Inoursubclass,wecanreferthedictionaryitself.Forexample:self[key].
Page22
1.4.4Files
1.4.4.1Whatfilesare
AfileisaPythonobjectthatgivesusaccesstoafileonthedisksystem.
Afileobjectcanbecreated("opened")forreading("r"mode),forwriting
("w"mode),orforappending("a"mode)toafile.
Openingafileforwritingerasesanexistingwiththatpath/name.Openinga
fileforappenddoesnot.
1.4.4.2Whentousefiles
Useafileobjectanytimeyouwishtoreadfromorwritetothediskfilesystem.
1.4.4.3Howtousefiles
Hereisanexamplethat(1)writestoafile,then(2)appendstothatfile,andfinally,
(3)readsfromthefile:
defwrite_file(outfilename):
outfile=open(outfilename,'w')
outfile.write('Line#1\n')
outfile.close()
defappend_file(outfilename):
outfile=open(outfilename,'a')
outfile.close()
defread_file(infilename):
infile=open(infilename,'r')
forlineininfile:
printline.rstrip()
infile.close()
deftest():
filename='temp_file.txt'
write_file(filename)
read_file(filename)
append_file(filename)
print''*50
read_file(filename)
test()
1.4.4.4ReadingTextFiles
Toreadatextfile,firstcreateafileobject.Hereisanexample:
inFile=open('messages.log','r')
Thenusethefileobjectasaniteratororuseoneormoreofthefileobject'smethods
Page23
toprocessthecontentsofthefile.Hereareafewstrategies:
Usefor line in inFile:toprocessonelineatatime.Youcandothisbecause

(atleastsincePython2.3)fileobjectsobeytheiteratorprotocol,thatisthey
supportmethods__iter__()andnext().Formoreontheiteratorprotocolsee
PythonStandardLibrary:IteratorTypes
http://docs.python.org/library/stdtypes.html#iteratortypes.
Example:
>>>inFile=file('tmp.txt','r')
>>>forlineininFile:
...print'Line:',line,
...
Line:aaaaa
Line:bbbbb
Line:ccccc
Line:ddddd
Line:eeeee
>>>inFile.close()
ForearlierversionsofPython,onestrategyistouse"inFile.readlines()",
whichcreatesalistoflines.
Ifyourwanttogetthecontentsofanentiretextfileasacollectionoflines,
usereadlines().Alternatively,youcoulduseread()followedbysplitlines().
Example:
>>>inFile=open('data2.txt','r')
>>>lines=inFile.readlines()
>>>inFile.close()
>>>lines
['aaabbbccc\n','dddeeefff\n','ggghhhiii\n']
>>>
>>>inFile=open('data2.txt','r')
>>>content=inFile.read()
>>>inFile.close()
>>>lines=content.splitlines()
>>>lines
['aaabbbccc','dddeeefff','ggghhhiii']
UseinFile.read()togettheentirecontentsofthefile(astring).Example:
>>>inFile=open('tmp.txt','r')
>>>content=inFile.read()
>>>inFile.close()
>>>printcontent
aaabbbccc
dddeeefff
ggghhhiii
>>>words=content.split()
>>>printwords
['aaa','bbb','ccc','ddd','eee','fff','ggg','hhh',
'iii']
>>>forwordinwords:
...printword
...
aaa
bbb
ccc
ddd
Page24
eee
fff
ggg
hhh
iii
1.5SimpleStatements
SimplestatementsinPythondonotcontainanestedblock.
1.5.1printstatement
Alert:InPythonversion3.0,theprintstatementhasbecometheprint()builtin
function.Youwillneedtoaddparentheses.
Theprintstatementsendsoutputtostdout.
Hereareafewexamples:
printobj
printobj1,obj2,obj3
print"Mynameis%s"%name
Notes:
Toprintmultipleitems,separatethemwithcommas.Theprintstatement
insertsablankbetweenobjects.
Theprintstatementautomaticallyappendsanewlinetooutput.Toprint
withoutanewline,addacommaafterthelastobject,oruse"sys.stdout",for
example:
print'Outputwithnonewline',
whichwillappendablank,or:
importsys
sys.stdout.write("Someoutput")
Toredefinethedestinationofoutputfromtheprintstatement,replace
sys.stdoutwithaninstanceofaclassthatsupportsthewritemethod.For
example:
importsys
classWriter:
def__init__(self,filename):
self.filename=filename
defwrite(self,msg):
f=file(self.filename,'a')
f.write(msg)
f.close()
sys.stdout=Writer('tmp.log')
print'Logmessage#1'
print'Logmessage#2'
print'Logmessage#3'
Page25
MoreinformationontheprintstatementisatTheprintstatement
http://docs.python.org/reference/simple_stmts.html#theprintstatement.
1.5.2Assignmentstatement
Theassignmentoperatoris=.
Herearesomeofthethingsyoucanassignavalueto:
Aname(variable)
Anitem(position)inalist.Example:
>>>a=[11,22,33]
>>>a
[11,22,33]
>>>a[1]=99
>>>a
[11,99,33]
Akeyinadictionary.Example:
>>>names={}
>>>names['albert']=25
>>>names
{'albert':25}
Asliceinalist.Example:
>>>a=[11,22,33,44,55,66,77,]
>>>a
[11,22,33,44,55,66,77]
>>>a[1:3]=[999,888,777,666]
>>>a
[11,999,888,777,666,44,55,66,77]
Atupleorlist.Assignmenttoatupleorlistperformsunpacking.Example:
>>>values=111,222,333
>>>values
(111,222,333)
>>>a,b,c=values
>>>a
111
>>>b
222
>>>c
333
Unpackingsuggestsaconvenientidiomforreturningandcapturingamultiple
argumentsfromafunction.Example:
>>>defmultiplier(n):
...returnn,n*2,n*3
...
>>>
>>>x,y,z=multiplier(4)
>>>x
4
>>>y
8
Page26
>>>z
12
Ifafunctionneedstoreturnavariablenumberofvalues,thenunpackingwill
notdo.But,youcanstillreturnmultiplevaluesbyreturningacontainerof
somekind(forexample,atuple,alist,adictionary,aset,etc.).
Anattribute.Example:
>>>classA(object):
...pass
...
>>>c=A()
>>>
>>>a=A()
>>>a.size=33
>>>printa.size
33
>>>a.__dict__
{'size':33}
1.5.3importstatement
Thingstoknowabouttheimportstatement:
Theimportstatementmakesamoduleanditscontentsavailableforuse.
Theimportstatementevaluatesthecodeinamodule,butonlythefirsttime
thatanygivenmoduleisimportedinanapplication.
Allmodulesinanapplicationthatimportagivenmoduleshareasinglecopy
ofthatmodule.Example:ifmodulesAandBbothimportmoduleC,thenA
andBshareasinglecopyofC.
Hereareseveralformsoftheimportstatement:
Importamodule.Refertoanattributeinthatmodule:
importtest
printtest.x
Importaspecificattributefromamodule:
fromtestimportx
fromothertestimporty,z
printx,y,z
Importalltheattributesinamodule:
fromtestimport*
printx
printy
Recommendation:Usethisformsparingly.Usingfrom mod import *makes

itdifficulttotrackdownvariablesand,thus,todebugyourcode
Importamoduleandrenameit.Importanattributefromamoduleandrename
it:
importtestastheTest
fromtestimportxastheValue
printtheTest.x
Page27
printtheValue
Afewcommentsaboutimport:
Theimportstatementalsoevaluatesthecodeintheimportedmodule.
But,thecodeinamoduleisonlyevaluatedthefirsttimeitisimportedina
program.So,forexample,ifamodulemymodule.pyisimportedfromtwo
othermodulesinaprogram,thestatementsinmymodulewillbeevaluated
onlythefirsttimeitisimported.
Ifyouneedevenmorevarietythattheimportstatementoffers,seetheimp
module.DocumentationisatimpAccesstheimportinternals
http://docs.python.org/library/imp.html#moduleimp.Alsoseethe
__import__()builtinfunction,whichyoucanreadabouthere:Builtin
Functions:__import()
http://docs.python.org/library/functions.html#__import__.
MoreinformationonimportisatLanguageReference:Theimportstatementhttp://
docs.python.org/reference/simple_stmts.html#theimportstatement.
1.5.4assertstatement
Usetheassertstatementtoplaceerrorcheckingstatementsinyourcode.Hereisan
example:
deftest(arg1,arg2):
arg1=float(arg1)
arg2=float(arg2)
assertarg2!=0,'Baddividendarg1:%farg2:%f'%(arg1,
arg2)
ratio=arg1/arg2
print'ratio:',ratio
Whenarg2iszero,runningthiscodewillproducesomethinglikethefollowing:
File"tmp.py",line22,in?
main()
File"tmp.py",line18,inmain
test(args[0],args[1])
File"tmp.py",line8,intest
assertarg2!=0,'Baddividendarg1:%farg2:%f'%(arg1,
arg2)
AssertionError:Baddividendarg1:2.000000arg2:0.000000
Afewcomments:
Noticethatthetracebackidentifiesthefileandlinewherethetestismadeand
showsthetestitself.
Ifyourunpythonwiththeoptimizeoptions(OandOO),theassertiontestis
notperformed.
Thesecondargumenttoassert()isoptional.
Page28
1.5.5globalstatement
TheproblemImagineaglobalvariableNAME.If,inafunction,thefirstmentionof
thatvariableis"name=NAME",thenI'llgetthevalueofthetheglobalvariable
NAME.But,if,inafunction,myfirstmentionofthatvariableisanassignmentto
thatvariable,thenIwillcreateanewlocalvariable,andwillnotrefertotheglobal
variableatall.Consider:
NAME="Peach"
defshow_global():
name=NAME
print'(show_global)name:%s'%name
defset_global():
NAME='Nectarine'
name=NAME
print'(set_global)name:%s'%name
show_global()
set_global()
show_global()
Runningthiscodeproduces:
(show_global)name:Peach
(set_global)name:Nectarine
Theset_globalmodifiesalocalvariableandnottheglobalvariableasImighthave
intended.
ThesolutionHowcanIfixthat?Hereishow:
NAME="Peach"
defshow_global():
name=NAME
print'(show_global)name:%s'%name
defset_global():
globalNAME
NAME='Nectarine'
name=NAME
print'(set_global)name:%s'%name
show_global()
set_global()
show_global()
Noticetheglobalstatementinfunctionset_global.Runningthiscodedoesmodifythe
globalvariableNAME,andproducesthefollowingoutput:
(set_global)name:Nectarine
(show_global)name:Nectarine
Comments:
Page29
Youcanlistmorethanoneveriableintheglobalstatement.Forexample:
globalNAME1,NAME2,NAME3
1.6CompoundstatmentsControlStructures
Acompoundstatementhasanested(andindented)blockofcode.Acompound
statementmayhavemultipleclauses,eachwithanestedblockofcode.Each
compoundstatementhasaheaderline(whichstartswithakeywordandendswitha
colon).
1.6.1if:statement
Theifstatementenablesustoexecutecode(ornot)dependingonacondition:
ifcondition:
statementblock
ifcondition:
statementblock1
else:
statementblock2
ifcondition1:
statementblock1
elifcondition2:
statementblock2
o
o
o
else:
statementblockn
Hereisanexample:
>>>y=25
>>>
>>>ify>15:
...print'yislarge'
...else:
...print'yissmall'
...
yislarge
Afewnotes:
Theconditioncanbeanyexpression,i.e.somethingthatreturnsavalue.A
detaileddescriptionofexpressionscanbefoundatPythonLanguage
Reference:Expressionshttp://docs.python.org/reference/expressions.html.
Parenthesesarenotneededaroundthecondition.Useparenthesestogroup
subexpressionsandcontroltheorderofevaluationwhenthenaturaloperator
precedenceisnotwhatyouwant.Python'soperatorprecedencesaredescribed
atPythonLanguageReference:Expressions:Summary
http://docs.python.org/reference/expressions.html#summary.
Pythonhasnoswitchstatement.Useif:elif:....Orconsiderusingadictionary.
Hereisanexamplethatusesbothofthesetechniques:
Page30
deffunction1():
print"Hi.I'mfunction1."
deffunction2():
deffunction3():
deferror_function():
print"Invalidoption."
deftest1():
while1:
code=raw_input('Enter"one","two","three",or
"quit":')
ifcode=='quit':
break
ifcode=='one':
function1()
elifcode=='two':
function2()
elifcode=='three':
function3()
else:
error_function()
deftest2():
mapper={'one':function1,'two':function2,'three':
function3}
while1:
code=raw_input('Enter"one","two","three",or
"quit":')
ifcode=='quit':
break
func=mapper.get(code,error_function)
func()
deftest():
test1()
print''*50
test2()
if__name__=='__main__':
test()
1.6.2for:statement
Thefor:statementenablesustoiterateovercollections.Itenablesustorepeatasetof
linesofcodeonceforeachiteminacollection.Collectionsarethingslikestrings
(arraysofcharacters),lists,tuples,anddictionaries.
Hereisanexample:
>>>collection=[111,222,333]
>>>foritemincollection:
...print'item:',item
...
item:111
item:222
item:333
Page31
Comments:
Youcaniterateoverstrings,lists,andtuples.Actually,youcaniterateover
almostanycontainerlikeobject.
Iterateoverthekeysorvaluesinadictionarywith"aDict.keys()"and
"aDict.values()".Hereisanexample:
>>>aDict={'cat':'furryandcute','dog':'friendlyand
smart'}
>>>aDict.keys()
['dog','cat']
>>>aDict.values()
['friendlyandsmart','furryandcute']
>>>forkeyinaDict.keys():
...print'A%sis%s.'%(key,aDict[key])
...
Adogisfriendlyandsmart.
Acatisfurryandcute.
InrecentversionsofPython,adictionaryitselfisaniteratorforitskeys.
Therefore,youcanalsodothefollowing:
>>>forkeyinaDict:
...print'A%sis%s.'%(key,aDict[key])
...
Adogisfriendlyandsmart.
Acatisfurryandcute.
And,inrecentversionsofPython,afileisalsoaniteratoroverthelinesinthe
file.Therefore,youcandothefollowing:
>>>forlineininfile:
...printline,
...
Thisisline#1
Thisisline#2
Thisisline#3
>>>infile.close()
Thereareotherkindsofiterators.Forexample,thebuiltiniterwillproduce
aniteratorfromacollection.Hereisanexample:
deftest():
anIter=iter([11,22,33])
foriteminanIter:
print'item:',item
test()
Whichproduces:
item:11
item:22
item:33
Youcanalsoimplementiteratorsofyourown.Onewaytodoso,definea
functionthatreturnsvalueswithyield(insteadofwithreturn).Hereisan
example:
Page32
deft(collection):
icollection=iter(collection)
foriteminicollection:
yield'||%s||'%item
deftest():
collection=[111,222,333,]
forxint(collection):
printx
test()
Whichprintsout:
||111||
||222||
||333||
1.6.2.1Thefor:statementandunpacking
Ifaniteratorproducesasequenceoflistsortuples,eachofwhichcontainthesame
(small)numberofitems,thenyoucandounpackingdirectlyintheheaderofthefor:
statement.Hereisanexample:
In[5]:collection=[('apple','red'),('banana','yello'),
('kiwi','green')]
In[6]:forname,colorincollection:
...:print'name:%s,color:%s'%(name,color,)
...:
...:
name:apple,color:red
name:banana,color:yello
name:kiwi,color:green
Theunpackingdescribedaboveandtheenumeratebuiltinfunctionprovidesa
convenientwaytoprocessitemsinasequencewithanindex.Thisexampleaddseach
valueinonelisttothecorrespondingvalueinasecondlist:
In[9]:a=[11,22,33]
In[10]:a=[11,22,33]
In[11]:
In[12]:a=[11,22,33]
In[13]:b=[111,222,333]
In[14]:foridx,valueinenumerate(a):
....:b[idx]+=value
....:
....:
In[15]:a
Out[15]:[11,22,33]
In[16]:b
Out[16]:[122,244,366]
Anotherwaytoimplementaniteratoristoimplementaclassthatsupportstheiterator
protocol.SeePythonStandardLibrary:IteratorTypes
http://docs.python.org/library/stdtypes.html#iteratortypesformoreonimplementing
iterators.ButnoticethatthisprotocolhaschangedinPython3.0.Forinformationon
theiteratorprotocolinPython3.0,seePython3.0StandardLibrary:IteratorTypes
Page33
http://docs.python.org/3.0/library/stdtypes.html#iteratortypes.
1.6.3while:statement
while:isanotherrepeatingstatement.Itexecutesablockofcodeuntilaconditionis
false.
Hereisanexample:
>>>reply='repeat'
>>>whilereply=='repeat':
...print'Hello'
...reply=raw_input('Enter"repeat"todoitagain:')
...
Hello
Enter"repeat"todoitagain:repeat
Hello
Enter"repeat"todoitagain:bye
Comments:
Usethebreakstatementtoexitimmediatelyfromaloop.Thisworksinboth
for:andwhile:.Hereisanexamplethatusesbreakinafor:statement:
#for_break.py
"""Countlinesuntilalinethatbeginswithadouble#.
"""
importsys
defcountLines(infilename):
infile=file(infilename,'r')
count=0
forlineininfile.readlines():
line=line.strip()
ifline[:2]=='##':
break
count+=1
returncount
defusage():
print'Usage:pythonpython_101_for_break.py
<infilename>'
sys.exit(1)
defmain():
args=sys.argv[1:]
iflen(args)!=1:
usage()
count=countLines(args[0])
print'count:',count
main()
Usethecontinuestatementtoskiptheremainderofthecodeblockinafor:or
while:statement.Acontinueisashortcircuitwhich,ineffect,branches
immediatelybacktothetopofthefor:orwhile:statement(orifyouprefer,to
Page34
theendoftheblock).
Thetestif __name__ == '__main__':isusedtoenableascripttobothbe(1)
importedand(2)runfromthecommandline.Thatconditionistrueonlywhen
thescriptisrun,butnotimported.ThisisacommonPythonidiom,whichyou
shouldconsiderincludingattheendofyourscripts,whether(1)togiveyour
usersademonstrationofwhatyourscriptdoesandhowtouseitor(2)to
provideatestofthescript.
1.6.4try:except:andraiseExceptions
Useatry:except:statementtocatchanexception.
Usetheraisestatementtoraiseanexception.
Commentsandhints:
Catchallexceptionswitha"bare"except:.Forexample:
>>>try:
...x=y
...except:
...print'ynotdefined'
...
ynotdefined
Note,however,thatitisusuallybettertocatchspecificexceptions.
Catchaspecificerrorbyreferingtoanexceptionclassintheexcept:.To
determinewhaterrororexceptionyouwanttocatch,generateitandtryit.
BecausePythonreportserrorswithawalkbackthatendswithreportingthe
exception,youcanlearnwhichexceptiontocatch.Forexample,supposeI
wanttolearnwhichexceptionisthrownwhenaPythoncan'topenafile.Ican
trythefollowingfromtheinteractiveprompt:
>>>myfile=file('amissingfile.py','r')
File"<stdin>",line1,in?
IOError:[Errno2]Nosuchfileordirectory:
'amissingfile.py'
So,nowIknowthatIcando:
deftest():
try:
myfile=file('amissingfile.py','r')
exceptIOError:
print'amissingfile.pyismissing'
test()
Whichproduces:
amissingfile.pyismissing
Exceptiontypesaredescribedhere:PythonStandardLibrary:Builtin
Exceptionshttp://docs.python.org/library/exceptions.html.
Catchanyoneofseveralexecptiontypesbyusingatuplecontainingthe
exceptionstobecaught.Example:
Page35
try:
f=open('abcdexyz.txt','r')
d={}
x=d['name']
except(IOError,KeyError),e:
print'Theerroris',e
Notethatmultipletypesofexceptionstobecaughtbyasingleexcept:clause
areinparentheses;theyareatuple.
Youcancustomizeyourerrorhandlingstillfurther(1)bypassinganobject
whenyouraisetheexceptionand(2)bycatchingthatobjectintheexcept:
clauseofyourtry:statement.Bydoingso,youcanpassinformationupfrom
theraisestatementtoanexceptionhandler.Onewayofdoingthisistopass
anobject.Areasonablestrategyistodefineasubclassofastandard
exception.Forexample:
classE(Exception):
def__init__(self,msg):
self.msg=msg
defgetMsg(self):
returnself.msg
deftest():
try:
raiseE('mytesterror')
exceptE,obj:
print'Msg:',obj.getMsg()
test()
Whichproduces:
Msg:mytesterror
Ifyoucatchanexceptionusingtry:except:,butthenfindthatyoudonotwant
tohandletheexceptionatthatlocation,youcan"reraise"thesameexception
(withthesamearguments)byusingraisewithnoarguments.Anexample:
classGeneralException(Exception):
pass
classSimpleException(GeneralException):
pass
classComplexException(GeneralException):
pass
defsome_func_that_throws_exceptions():
#raiseSimpleException('thisisasimpleerror')
raiseComplexException('thisisacomplexerror')
deftest():
try:
some_func_that_throws_exceptions()
exceptGeneralException,e:
ifisinstance(e,SimpleException):
printe
else:
raise
Page36
test()
1.7Organization
ThissectiondescribesPythonfeaturesthatyoucanusetoorganizeandstructureyour
code.
1.7.1Functions
1.7.1.1Abasicfunction
Usedeftodefineafunction.Hereisasimpleexample:
deftest(msg,count):
foridxinrange(count):
print'%s%d'%(msg,idx)
test('Test#',4)
Comments:
Afterevaluationdefcreatesafunctionobject.
Callthefunctionusingtheparenthesesfunctioncallnotation,inthiscase
"test('Test#',4)".
AswithotherPythonobjects,youcanstuffafunctionobjectintoother
structuressuchastuples,lists,anddictionaries.Hereisanexample:
#Createatuple:
val=(test,'Alabel:',5)
#Callthefunction:
val[0](val[1],val[2])
1.7.1.2Afunctionwithdefaultarguments
Providingdefaultargumentsallowsthecallertoomitsomearguments.Hereisan
example:
deftestDefaultArgs(arg1='default1',arg2='default2'):
print'arg1:',arg1
print'arg2:',arg2
testDefaultArgs('Explicitvalue')
Theaboveexampleprints:
arg1:Explicitvalue
arg2:default2
1.7.1.3Argumentlistsandkeywordargumentlists
Hereisanexample:
Page37
deftestArgLists_1(*args,**kwargs):
print'args:',args
print'kwargs:',kwargs
testArgLists_1('aaa','bbb',arg1='ccc',arg2='ddd')
deftestArgLists_2(arg0,*args,**kwargs):
print'arg0:"%s"'%arg0
print'args:',args
deftest():
testArgLists_1('aaa','bbb',arg1='ccc',arg2='ddd')
print'='*40
testArgLists_2('afirstargument','aaa','bbb',arg1='ccc',
arg2='ddd')
test()
Runningthisexampledisplays:
args:('aaa','bbb')
kwargs:{'arg1':'ccc','arg2':'ddd'}
========================================
arg0:"afirstargument"
args:('aaa','bbb')
Alittleguidance:
Positionalargumentsmustproceedallkeywordargumentswhenyoucallthe
function.
Youcanalsohave"normal"argumentsinthefunctiondefinition.For
example:def test(arg0, *args, **kwargs):.Seethesecondexampleabove.
Thevalueofthekeywordparameter(**kwargs)isadictionary,soyoucando
anythingwithitthatyoudowithanormaldictionary.
1.7.1.4Callingafunctionwithkeywordarguments
Youcanalsocallafunctionusingthenameofaparameterasakeyword.Hereisan
example:
deftest_keyword_args(foreground_color='black',
background_color='white',
link_color='blue',
visited_link_color='red'):
print'foreground_color:"%s"'%foreground_color
print'background_color:"%s"'%background_color
print'link_color:"%s"'%link_color
print'visited_link_color:"%s"'%visited_link_color
deftest():
test_keyword_args()
print''*40
test_keyword_args(background_color='green')
print''*40
test_keyword_args(link_color='gray',
visited_link_color='yellow')
Page38
test()
Whenwerunthisexample,itproducesthefollowing:
foreground_color:"black"
background_color:"white"
link_color:"blue"
visited_link_color:"red"
background_color:"green"
link_color:"blue"
visited_link_color:"red"
background_color:"white"
link_color:"gray"
visited_link_color:"yellow"
1.7.2Classesandinstances
1.7.2.1Abasicclass
Defineabasicclassasfollows:
classBasic:
def__init__(self,name):
self.name=name
defshow(self):
print'Basicname:%s'%self.name
deftest():
obj1=Basic('Apricot')
obj1.show()
test()
Runningtheaboveexampleproducesthefollowing:
Basicname:Apricot
Explanation:
Methodsareaddedtotheclasswithdef.Thefirstargumenttoamethodisthe
classinstance.Byconventionitisspelled"self".
Theconstructorforaclassisamethodnamed__init__.
Theselfvariablemustbeexplicitlylistedasthefirstargumenttoamethod.
Youcouldspellitdifferentlyfrom"self",butdon'tdoso.
Instancevariablesarereferredtowith"self.XXX".Noticehowinourexample
anargumenttotheconstructorissavedasaninstancevariable.
Aninstanceiscreatedby"calling"theclass.Forexample:obj =
Basic('Apricot').
Inadditionto__init__thereareotherspecialmethodnamesoftheform
Page39
"__XXX__",whichareusedtocustomizeclassesandtheirinstances.These
aredescribedatPythonLanguageReference:Specialmethodnames
http://docs.python.org/reference/datamodel.html#specialmethodnames
<http://docs.python.org/reference/datamodel.html#specialmethodnames>_.
Afewmorenotesonself:
selfisareferencetotheinstance.Thinkofit(inpart)asareferencetothe
containerforthedataorstatefortheobject.
Inmanyobjectorientedprogramminglanguages,theinstanceishiddeninthe
methoddefinitions.Theselanguagestypicallyexplainthisbysaying
somethinglike"Theinstanceispassedasanimplicitfirstargumenttothe
method."
InPython,theinstanceisvisibleandexplicitinmethoddefinitions.Youmust
explicitlydeclaretheinstanceasthefirstparameterofeach(instance)method.
Thisfirstparameteris(almost)alwaysspelled"self".
1.7.2.2Inheritance
DefineaclassSpecialthatinheritsfromasuperclassBasicasfollows:
classBasic:
self.name=name
defshow(self):
print'Basicname:%s'%self.name
classSpecial(Basic):
def__init__(self,name,edible):
Basic.__init__(self,name)
self.upper=name.upper()
self.edible=edible
defshow(self):
Basic.show(self)
print'Specialuppername:%s.'%self.upper
ifself.edible:
print"It'sedible."
else:
print"It'snotedible."
defedible(self):
returnself.edible
deftest():
obj1=Basic('Apricot')
obj1.show()
print'='*30
obj2=Special('Peach',1)
obj2.show()
test()
Runningthisexampleproducesthefollowing:
Basicname:Apricot
==============================
Basicname:Peach
Specialuppername:PEACH.It'sedible.
Page40
Comments:
Thesuperclassislistedaftertheclassnameinparentheses.Formultiple
inheritence,separatethesuperclasseswithcommas.
Callamethodinthesuperclass,bypassingthemethodwiththesamenamein
thesubclass,fromthesubclassbyusingthesuperclassname.Forexample:
Basic.__init__(self, name)andBasic.show(self).
Inourexample(above),thesubclass(Special)specializesthesuperclass
(Basic)byaddingadditionalmembervariables(self.upperandself.edible)
andbyaddinganadditionalmethod(edible).
1.7.2.3Classdata
Aclassdatamemberisamemberthathasonlyonevaluefortheclassandallits
instances.HereisanexamplefromthePythonFAQat
http://www.python.org/doc/FAQ.html:
classC:
count=0#numberoftimesC.__init__called
def__init__(self):
C.count+=1
defgetcount(self):
returnC.count#orreturnself.count
deftest():
c1=C()
print'Currentcount:',c1.getcount()
c2=C()
print'Currentcount:',c2.getcount()
test()
Runningthisexampleproduces:
Currentcount:1
Currentcount:2
1.7.2.4Staticmethodsandclassmethods
Newstyleclassescanhavestaticmethodsandclassmethods.
Anewstyleclassisaclassthatinheritsdirectlyorindirectlyfromobjectorfroma
builtintype.
Hereisanexamplethatshowshowtodefinestaticmethodsandclassmethods:
classAdvanced(object):
self.name=name
defDescription():
return'Thisisanadvancedclass.'
defClassDescription(cls):
return'Thisisadvancedclass:%s'%repr(cls)
Description=staticmethod(Description)
ClassDescription=classmethod(ClassDescription)
Page41
obj1=Advanced('Nectarine')
printobj1.Description()
printobj1.ClassDescription()
print'='*30
printAdvanced.Description()
printAdvanced.ClassDescription()
Runningtheaboveproducesthefollowingoutput:
Thisisanadvancedclass.
Thisisadvancedclass:<class__main__.Advancedat0x401c926c>
==============================
Thisisanadvancedclass.
Thisisadvancedclass:<class__main__.Advancedat0x401c926c>
Notes:
Theclassinheritsfromclassobject,whichmakesitanewstyleclass.
Createastaticmethodwithx = staticmethod(y),whereyisanormalmethod
butwithouttheself/firstparameter.
Createaclassmethodwithx = classmethod(y),whereyisanormalmethod.
Thedifferencebetweenstaticandclassmethodsisthataclassmethodreceivesthe
class(nottheinstance)asitsfirstargument.Asummary:
Anormal/standardmethodalwaysreceivesaninstanceasitsfirstargument.
Aclassmethodalwaysreceivestheclassasitsfirstargument.
Astaticmethoddoesnot(automatically)receiveeithertheinstanceorthe
classasthefirstargument.
Youcancallstaticandclassmethodsusingeitheraninstanceoraclass.Inour
exampleeither"obj1.Description()"or"Advanced.Description()"willwork.
YoushouldalsoreviewtherelevantstandardPythondocumentationonthe
classmethodandstaticmethodbuiltinfunctions,whichyoucanfindatPython
LibraryReference2.1BuiltinFunctions
http://docs.python.org/library/functions.html.
Bynow,youarelikelytobeasking:"WhyandwhenshouldIuseclassmethodsand
staticmethods?"Hereisabitofguidance:
Mostofthetime,almostalways,implementplaininstancemethods.
Implementaninstancemethodwheneverthemethodneedsaccesstothe
valuesthatarespecifictotheinstanceorneedstocallothermethodsthathave
accesstoinstancespecificvalues.Ifthemethodneedsself,thenyouprobably
needaninstancemethod.
Implementaclassmethod(1)whenthemethoddoesnotneedaccessto
instancevariablesand(2)whenyoudonotwanttorequirethecallerofthe
methodtocreateaninstanceand(3)whenthemethodneedsaccesstoclass
variables.Aclassmethodmaybecalledoneitheraninstanceortheclass.A
classmethodgetstheclassasafirstargument,whetheritiscalledontheclass
ortheinstance.Ifthemethodneedsaccesstotheclassbutdoesnotneedself,
thenthinkclassmethod.
Implementastaticmethodifyoumerelywanttoputthecodeofthemethod
Page42
withinthescopeoftheclass,perhapsforpurposesoforganizingyourcode,
butthemethodneedsaccesstoneitherclassnorinstancevariables(though
youcanaccessclassvariablesthroughtheclassitself).Astaticmethodmaybe
calledoneitheraninstanceortheclass.Astaticmethodgetsneithertheclass
northeinstanceasanargument.
Tosummarize:
Implementaninstancemethod,unless...themethodneedsaccesstoclassvariables
butnotinstancevariables,thenimplementaclassmethod,unless...themethodneeds
accesstoneitherinstancevariablesnorclassvariablesandyoustillwanttoincludeit
withintheclassdefinition,thenimplementastaticmethod.
Aboveall,writeclear,plaincodethatwillbeunderstandabletoyourreaders.Donot
useamoreconfusinglanguagefeatureanddonotforceyourreaderstolearnanew
languagefeatureunlesyouhaveagoodreason.
1.7.2.5Properties
Anewstyleclasscanhaveproperties.Apropertyisanattributeofaclassthatis
associatedwithagetterandasetterfunction.
Declarethepropertyanditsgetterandsetterfunctionswithproperty().
Hereisanexample:
classA(object):
count=0
self.name=name
defset_name(self,name):
print'settingname:%s'%name
self.name=name
defget_name(self):
print'gettingname:%s'%self.name
returnself.name
objname=property(get_name,set_name)
deftest():
a=A('apple')
print'name:%s'%a.objname
a.objname='banana'
print'name:%s'%a.objname
test()
gettingname:apple
name:apple
settingname:banana
gettingname:banana
name:banana
Notes:
Theclassinheritsfromclassobject,whichmakesitanewstyleclass.
Page43
Whenavalueisassignedtoaproperty,thesettermethodiscalled.
Whenthevalueofapropertyisaccessed,thegettermethodiscalled.
Youcanalsodefineadeletemethodandadocumentationattributefora
property.Formoreinformation,visit2.1BuiltinFunctionsandlookfor
property.
1.7.3Modules
YoucanuseamoduletoorganizeanumberofPythondefinitionsinasinglefile.A
definitioncanbeafunction,aclass,oravariablecontaininganyPythonobject.Here
isanexample:
#python_101_module_simple.py
"""
Thissimplemodulecontainsdefinitionsofaclassandseveral
functions.
"""
LABEL='=====Testingasimplemodule====='
classPerson:
"""Sampleofasimpleclassdefinition.
"""
def__init__(self,name,description):
self.name=name
self.description=description
defshow(self):
print'Personname:%sdescription:%s'%(self.name,
self.description)
deftest(msg,count):
"""Asampleofasimplefunction.
"""
foridxinrange(count):
print'%s%d'%(msg,idx)
deftestDefaultArgs(arg1='default1',arg2='default2'):
"""Afunctionwithdefaultarguments.
"""
print'arg1:',arg1
print'arg2:',arg2
deftestArgLists(*args,**kwargs):
"""
Afunctionwhichreferencestheargumentlistandkeyword
arguments.
"""
print'args:',args
defmain():
"""
Atestharnessforthismodule.
"""
printLABEL
person=Person('Herman','Acuteguy')
Page44
person.show()
print'='*30
test('Test#',4)
print'='*30
testDefaultArgs('Explicitvalue')
print'='*30
testArgLists('aaa','bbb',arg1='ccc',arg2='ddd')
main()
=====Testingasimplemodule=====
Personname:Hermandescription:Acuteguy
==============================
Test#0
Test#1
Test#2
Test#3
==============================
arg1:Explicitvalue
arg2:default2
==============================
args:('aaa','bbb')
Comments:
Thestringdefinitionsatthebeginningofeachofthemodule,classdefinitions,and
functiondefinitionsserveasdocumentationfortheseitems.Youcanshowthis
documentationwiththefollowingfromthecommandline:
$pydocpython_101_module_simple
Orthis,fromthePythoninteractiveprompt:
>>>importpython_101_module_simple
>>>help(python_101_module_simple)
Itiscommonanditisagoodpracticetoincludeatestharnessforthemoduleatthe
endofthesourcefile.Notethatthetest:
willbetrueonlywhenthefileisrun(e.g.fromthecommandlinewithsomething
like:
"$pythonpython_101_module_simple.py
butnotwhenthemoduleisimported.
Rememberthatthecodeinamoduleisonlyevaluatedthefirsttimeitisimportedina
program.So,forexample,changethevalueofaglobalvariableinamodulemight
causebehaviorthatusersofthemodulemightnotexpect.
Constants,ontheotherhand,aresafe.Aconstant,inPython,isavariablewhose
valueisinitializedbutnotchanged.AnexampleisLABEL,above.
Page45
1.7.4Packages
Apackageisawaytoorganizeanumberofmodulestogetherasaunit.Python
packagescanalsocontainotherpackages.
Togiveusanexampletotalkabout,considerthefollowpackagestructure:
package_example/
package_example/__init__.py
package_example/module1.py
package_example/module2.py
package_example/A.py
package_example/B.py
And,herearethecontents:
__init__.py:
#__init__.py
#Exposedefinitionsfrommodulesinthispackage.
frommodule1importclass1
frommodule2importclass2
module1.py:
#module1.py
classclass1:
def__init__(self):
self.description='class#1'
defshow(self):
printself.description
module2.py:
#module2.py
classclass2:
def__init__(self):
self.description='class#2'
defshow(self):
printself.description
A.py:
#A.py
importB
B.py:
#B.py
deffunction_b():
print'Hellofromfunction_b'
InordertobeusedasaPythonpackage(e.g.sothatmodulescanbeimportedfromit)
adirectorymustcontainafilewhosenameis__init__.py.Thecodeinthismoduleis
evaluatedthefirsttimeamoduleisimportedfromthepackage.
Inordertoimportmodulesfromapackage,youmayeitheraddthepackagedirectory
Page46
tosys.pathor,iftheparentdirectoryisonsys.path,usedotnotationtoexplicitly
specifythepath.Inourexample,youmightuse:"importpackage_example.module1".
Amoduleinapackagecanimportanothermodulefromthesamepackagedirectly
withoutusingthepathtothepackage.Forexample,themoduleAinoursample
packagepackage_examplecanimportmoduleBinthesamepackagewith"import
B".ModuleAdoesnotneedtouse"importpackage_example.B".
Youcanfindadditionalinformationonpackagesat
http://www.python.org/doc/essays/packages.html.
Suggestedtechniques:
Inthe__init__.pyfile,importandmakeavailableobjectsdefinedinmodulesinthe
package.Oursamplepackagepackage_exampledoesthis.Then,youcanusefrom
package_example import *toimportthepackageanditscontents.Forexample:
>>>frompackage_exampleimport*
>>>dir()
['__builtins__','__doc__','__file__','__name__',
'atexit','class1','class2','module1','module2',
'readline','rlcompleter','sl','sys']
>>>
>>>c1=class1()
>>>c2=class2()
>>>c1.show()
class#1
>>>c2.show()
class#2
Afewadditionalnotes:
WithPython2.3,youcancollectthemodulesinapackageintoaZipfileby
usingPyZipFilefromthePythonstandardlibrary.Seehttp://www.python.org/
doc/current/lib/pyzipfileobjects.html.
>>>importzipfile
>>>a=zipfile.PyZipFile('mypackage.zip','w',
zipfile.ZIP_DEFLATED)
>>>a.writepy('Examples')
>>>a.close()
Thenyoucanimportandusethisarchivebyinsertingitspathinsys.path.In
thefollowingexample,class_basic_1isamodulewithinpackagemypackage:
>>>importsys
>>>sys.path.insert(0,'/w2/Txt/Training/mypackage.zip')
>>>importclass_basic_1
Basicname:Apricot
>>>obj=class_basic_1.Basic('Wilma')
>>>obj.show()
Basicname:Wilma
1.8AcknowledgementsandThanks
ThankstotheimplementorsofPythonforproducinganexceptionallyusableand
enjoyableprogramminglanguage.
Page47
1.9SeeAlso
ThemainPythonWebSitehttp://www.python.orgformoreinformationon
Python.
ThePythondocumentationpagehttp://www.python.org/doc/forlotsof
documentationonPython.
Dave'sWebSitehttp://www.rexx.com/~dkuhlmanformoresoftwareand
informationonusingPythonforXMLandtheWeb.
Page48
2Part2AdvancedPython
2.1IntroductionPython201(Slightly)AdvancedPython
Topics
Thisdocumentisintendedasnotesforacourseon(slightly)advancedPythontopics.
2.2RegularExpressions
Formorehelponregularexpressions,see:
reRegularexpressionoperationshttp://docs.python.org/library/re.html
RegularExpressionHOWTOhttp://docs.python.org/howto/regex.html
2.2.1Definingregularexpressions
Aregularexpressionpatternisasequenceofcharactersthatwillmatchsequencesof
charactersinatarget.
Thepatternsorregularexpressionscanbedefinedasfollows:
Literalcharactersmustmatchexactly.Forexample,"a"matches"a".
Concatenatedpatternsmatchconcatenatedtargets.Forexample,"ab"("a"
followedby"b")matches"ab".
Alternatepatterns(separatedbyaverticalbar)matcheitherofthealternative
patterns.Forexample,"(aaa)|(bbb)"willmatcheither"aaa"or"bbb".
Repeatingandoptionalitems:
"abc*"matches"ab"followedbyzeroormoreoccurancesof"c",for
example,"ab","abc","abcc",etc.
"abc+"matches"ab"followedbyoneormoreoccurancesof"c",for
example,"abc","abcc",etc,butnot"ab".
"abc?"matches"ab"followedbyzerooroneoccurancesof"c",for
example,"ab"or"abc".
SetsofcharactersCharactersandsequencesofcharactersinsquarebrackets
formaset;asetmatchesanycharacterinthesetorrange.Forexample,
"[abc]"matches"a"or"b"or"c".And,forexample,"[_az09]"matchesan
underscoreoranylowercaseletteroranydigit.
GroupsParenthesesindicateagroupwithapattern.Forexample,
Page49
"ab(cd)*ef"isapatternthatmatches"ab"followedbyanynumberof
occurancesof"cd"followedby"ef",forexample,"abef","abcdef",
"abcdcdef",etc.
Therearespecialnamesforsomesetsofcharacters,forexample"\d"(any
digit),"\w"(anyalphanumericcharacter),"\W"(anynonalphanumeric
character),etc.Moremoreinformation,seePythonLibraryReference:
RegularExpressionSyntaxhttp://docs.python.org/library/re.html#regular
expressionsyntax
Becauseoftheuseofbackslashesinpatterns,youareusuallybetteroffdefining
regularexpressionswithrawstrings,e.g.r"abc".
2.2.2Compilingregularexpressions
Whenaregularexpressionistobeusedmorethanonce,youshouldconsider
compilingit.Forexample:
importsys,re
pat=re.compile('aa[bc]*dd')
while1:
line=raw_input('Enteraline("q"toquit):')
ifline=='q':
break
ifpat.search(line):
print'matched:',line
else:
print'nomatch:',line
Comments:
Weimportmodulereinordertouseregularexpresions.
re.compile()compilesaregularexpressionsothatwecanreusethecompiled
regularexpressionwithoutcompilingitrepeatedly.
2.2.3Usingregularexpressions
Usematch()tomatchatthebeginningofastring(ornotatall).
Usesearch()tosearchastringandmatchthefirststringfromtheleft.
Herearesomeexamples:
>>>importre
>>>pat=re.compile('aa[09]*bb')
>>>x=pat.match('aa1234bbccddee')
>>>x
<_sre.SRE_Matchobjectat0x401e9608>
>>>x=pat.match('xxxxaa1234bbccddee')
>>>x
>>>type(x)
<type'NoneType'>
>>>x=pat.search('xxxxaa1234bbccddee')
>>>x
Page50
Notes:
Whenamatchorsearchissuccessful,itreturnsamatchobject.Whenitfails,
itreturnsNone.
Youcanalsocallthecorrespondingfunctionsmatchandsearchinthere
module,e.g.:
>>>x=re.search(pat,'xxxxaa1234bbccddee')
>>>x
Foralistoffunctionsintheremodule,seeModuleContents
http://docs.python.org/library/re.html#modulecontents.
2.2.4Usingmatchobjectstoextractavalue
Matchobjectsenableyoutoextractmatchedsubstringsafterperformingamatch.A
matchobjectisreturnedbysuccessfulmatch.Thepartofthetargetavailableinthe
matchobjectistheportionmatchedbygroupsinthepattern,thatistheportionofthe
patterninsideparentheses.Forexample:
In[69]:mo=re.search(r'height:(\d*)width:(\d*)','height:
123width:456')
In[70]:mo.groups()
Out[70]:('123','456')
Hereisanotherexample:
importsys,re
Targets=[
'Thereare<<25>>sparrows.',
'Isee<<15>>finches.',
'Thereisnothinghere.',
]
deftest():
pat=re.compile('<<([09]*)>>')
forlineinTargets:
mo=pat.search(line)
ifmo:
value=mo.group(1)
print'value:%s'%value
else:
print'nomatch'
test()
Whenweruntheabove,itprintsoutthefollowing:
value:25
value:15
nomatch
Explanation:
Intheregularexpression,putparenthesesaroundtheportionoftheregular
expressionthatwillmatchwhatyouwanttoextract.Eachpairofparentheses
Page51
marksoffagroup.
Afterthesearch,checktodetermineiftherewasasuccessfulmatchby
checkingforamatchingobject."pat.search(line)"returnsNoneifthesearch
fails.
Ifyouspecifymorethanonegroupinyourregularexpression(morethatone
pairofparentheses),thenyoucanuse"value=mo.group(N)"toextractthe
valuematchedbytheNthgroupfromthematchingobject."value=
mo.group(1)"returnsthefirstextractedvalue;"value=mo.group(2)"returns
thesecond;etc.Anargumentof0returnsthestringmatchedbytheentire
regularexpression.
Inaddition,youcan:
Use"values=mo.groups()"togetatuplecontainingthestringsmatchedbyall
groups.
Use"mo.expand()"tointerpolatethegroupvaluesintoastring.Forexample,
"mo.expand(r'value1:\1value2:\2')"insertsthevaluesofthefirstandsecond
groupintoastring.Ifthefirstgroupmatched"aaa"andthesecondmatched
"bbb",thenthisexamplewouldproduce"value1:aaavalue2:bbb".For
example:
In[76]:mo=re.search(r'h:(\d*)w:(\d*)','h:123w:
456')
In[77]:mo.expand(r'Height:\1Width:\2')
Out[77]:'Height:123Width:456'
2.2.5Extractingmultipleitems
Youcanextractmultipleitemswithasinglesearch.Hereisanexample:
importsys,re
pat=re.compile('aa([09]*)bb([09]*)cc')
while1:
ifline=='q':
break
mo=pat.search(line)
ifmo:
value1,value2=mo.group(1,2)
print'value1:%svalue2:%s'%(value1,value2)
else:
print'nomatch'
Comments:
Usemultipleparenthesizedsubstringsintheregularexpressiontoindicatethe
portions(groups)tobeextracted.
"mo.group(1,2)"returnsthevaluesofthefirstandsecondgroupinthestring
matched.
Wecouldalsohaveused"mo.groups()"toobtainatuplethatcontainsboth
values.
Yetanotheralternativewouldhavebeentousethefollowing:print
Page52
mo.expand(r'value1: \1 value2: \2').
2.2.6Replacingmultipleitems
Asimplewaytoperformmultiplereplacementsusingaregularexpressionistouse
there.subn()function.Hereisanexample:
In[81]:re.subn(r'\d+','***','thereare203birdssittingin2
trees')
Out[81]:('thereare***birdssittingin***trees',2)
Formorecomplexreplacements,useafunctioninsteadofaconstantreplacement
string:
importre
defrepl_func(mo):
s1=mo.group(1)
s2='*'*len(s1)
returns2
deftest():
pat=r'(\d+)'
in_str='thereare2034birdsin21trees'
out_str,count=re.subn(pat,repl_func,in_str)
print'in:"%s"'%in_str
print'out:"%s"'%out_str
print'count:%d'%count
test()
Andwhenweruntheabove,itproduces:
in:"thereare2034birdsin21trees"
out:"thereare****birdsin**trees"
count:2
Notes:
Thereplacementfunctionreceivesoneargument,amatchobject.
There.subn()functionreturnsatuplecontainingtwovalues:(1)thestring
afterreplacementsand(2)thenumberofreplacementsperformed.
HereisanevenmorecomplexexampleYoucanlocatesubstrings(slices)ofa
matchandreplacethem:
importsys,re
pat=re.compile('aa([09]*)bb([09]*)cc')
while1:
ifline=='q':
break
mo=pat.search(line)
ifmo:
value1,value2=mo.group(1,2)
start1=mo.start(1)
Page53
end1=mo.end(1)
start2=mo.start(2)
end2=mo.end(2)
print'value1:%sstart1:%dend1:%d'%(value1,
start1,end1)
print'value2:%sstart2:%dend2:%d'%(value2,
start2,end2)
repl1=raw_input('Enterreplacement#1:')
repl2=raw_input('Enterreplacement#2:')
newline=(line[:start1]+repl1+line[end1:start2]+
repl2+line[end2:])
print'newline:%s'%newline
else:
print'nomatch'
Explanation:
Alternatively,use"mo.span(1)"insteadof"mo.start(1)"and"mo.end(1)"in
ordertogetthestartandendofasubmatchinasingleoperation.
"mo.span(1)"returnsatuple:(start,end).
Puttogetheranewstringwithstringconcatenationfrompiecesoftheoriginal
stringandreplacementvalues.Youcanusestringslicestogetthesubstrings
oftheoriginalstring.Inourcase,thefollowinggetsthestartofthestring,
addsthefirstreplacement,addsthemiddleoftheoriginalstring,addsthe
secondreplacement,andfinally,addsthelastpartoftheoriginalstring:
newline=line[:start1]+repl1+line[end1:start2]+repl2
+line[end2:]
Youcanalsousethesubfunctionormethodtodosubstitutions.Hereisanexample:
importsys,re
pat=re.compile('[09]+')
print'Replacingdecimaldigits.'
while1:
target=raw_input('Enteratargetline("q"toquit):')
iftarget=='q':
break
repl=raw_input('Enterareplacement:')
result=pat.sub(repl,target)
print'result:%s'%result
Hereisanotherexampleoftheuseofafunctiontoinsertcalculatedreplacements.
importsys,re,string
pat=re.compile('[am]+')
defreplacer(mo):
returnstring.upper(mo.group(0))
print'Uppercasingam.'
while1:
iftarget=='q':
break
Page54
result=pat.sub(replacer,target)
Notes:
Ifthereplacementargumenttosubisafunction,thatfunctionmusttakeone
argument,amatchobject,andmustreturnthemodified(orreplacement)
value.Thematchedsubstringwillbereplacedbythevaluereturnedbythis
function.
Inourcase,thefunctionreplacerconvertsthematchedvaluetouppercase.
Thisisalsoaconvenientuseforalambdainsteadofanamedfunction,forexample:
importsys,re,string
pat=re.compile('[am]+')
print'Uppercasingam.'
while1:
iftarget=='q':
break
result=pat.sub(
lambdamo:string.upper(mo.group(0)),
target)
2.3IteratorObjects
Note1:YouwillneedasufficientlyrecentversionofPythoninordertouseiterators
andgenerators.IbelievethattheywereintroducedinPython2.2.
Note2:TheiteratorprotocolhaschangedslightlyinPythonversion3.0.
Goalsforthissection:
Learnhowtoimplementageneratorfunction,thatis,afunctionwhich,when
called,returnsaniterator.
Learnhowtoimplementaclasscontainingageneratormethod,thatis,a
methodwhich,whencalled,returnsaniterator.
Learntheiteratorprotocol,specificallywhatmethodsaniteratormustsupport
andwhatthosemethodsmustdo.
Learnhowtoimplementaniteratorclass,thatis,aclasswhoseinstancesare
iteratorobjects.
Learnhowtoimplementrecursiveiteratorgenerators,thatis,aniterator
generatorwhichrecursivelyproducesiteratorgenerators.
Learnthatyourimplementationofaniteratorobject(aniteratorclass)can
"refresh"itselfandlearnatleastonewaytodothis.
Definitions:
IteratorAnditeratorisanobjectthatsatisfies(implements)theiterator
protocol.
IteratorprotocolAnobjectimplementstheiteratorprotocolifitimplements
bothanext()andan__iter__()methodwhichsatisfytheserules:(1)the
Page55
__iter__()methodmustreturntheiterator;(2)thenext()methodshould
returnthenextitemtobeiteratedoverandwhenfinished(therearenomore
items)shouldraisetheStopIterationexception.Theiteratorprotocolis
describedatIteratorTypes
IteratorclassAclassthatimplements(satisfies)theiteratorprotocol.In
particular,theclassimplementsnext()and__iter__()methodsasdescribed
aboveandinIteratorTypes
(Iterator)generatorfunctionAfunction(ormethod)which,whencalled,
returnsaniteratorobject,thatis,anobjectthatsatisfiestheiteratorprotocol.A
functioncontainingayieldstatementautomaticallybecomesagenerator.
GeneratorexpressionAnexpressionwhichproducesaniteratorobject.
Generatorexpressionshaveaformsimilartoalistcomprehension,butare
enclosedinparenthesesratherthansquarebrackets.Seeexamplebelow.
Afewadditionalbasicpoints:
Afunctionthatcontainsayieldstatementisageneratorfunction.When
called,itreturnsaniterator,thatis,anobjectthatprovidesnext()and
__iter__()methods.
Theiteratorprotocolisdescribedhere:PythonStandardLibrary:Iterator
Typeshttp://docs.python.org/library/stdtypes.html#iteratortypes.
Aclassthatdefinesbothanext()methodanda__iter__()methodsatisfies
theiteratorprotocol.So,instancesofsuchaclasswillbeiterators.
Pythonprovidesavarietyofwaystoproduce(implement)iterators.This
sectiondescribesafewofthoseways.Youshouldalsolookattheiter()built
infunction,whichisdescribedinThePythonStandardLibrary:Builtin
Functions:iter()http://docs.python.org/library/functions.html#iter.
Aniteratorcanbeusedinaniteratorcontext,forexampleinaforstatement,
inalistcomprehension,andinageneratorexpression.Whenaniteratoris
usedinaniteratorcontext,theiteratorproducesitsvalues.
Thissectionattemptstoprovideexamplesthatillustratethegenerator/iteratorpattern.
Whyisthisimportant?
Oncemastered,itisasimple,convenient,andpowerfulprogrammingpattern.
Ithasmanyandpervasiveuses.
Ithelpstolexicallyseparatetheproducercodefromtheconsumercode.Doing
somakesiteasiertolocateproblemsandtomodifyorfixcodeinawaythatis
localizedanddoesnothaveunwantedsideeffects.
Implementingyourowniterators(andgenerators)enablesyoutodefineyour
ownabstractsequences,thatis,sequenceswhosecompositionaredefinedby
yourcomputationsratherthanbytheirpresenceinacontainer.Infact,your
iteratorcancalculateorretrievevaluesaseachoneisrequested.
ExamplesTheremainderofthissectionprovidesasetofexampleswhichimplement
anduseiterators.
Page56
2.3.1ExampleAgeneratorfunction
Thisfunctioncontainsayieldstatement.Therefore,whenwecallit,itproducesan
iterator:
defgenerateItems(seq):
foriteminseq:
yield'item:%s'%item
anIter=generateItems([])
print'dir(anIter):',dir(anIter)
anIter=generateItems([111,222,333])
forxinanIter:
printx
anIter=generateItems(['aaa','bbb','ccc'])
printanIter.next()
printanIter.next()
printanIter.next()
printanIter.next()
Runningthisexampleproducesthefollowingoutput:
dir(anIter):['__class__','__delattr__','__doc__',
'__getattribute__',
'__hash__','__init__','__iter__','__new__','__reduce__',
'__reduce_ex__','__repr__','__setattr__','__str__','gi_frame',
'gi_running','next']
item:111
item:222
item:333
item:aaa
item:bbb
item:ccc
File"iterator_generator.py",line14,in?
printanIter.next()
StopIteration
Notesandexplanation:
Thevaluereturnedbythecalltothegenerator(function)isaniterator.It
obeystheiteratorprotocol.Thatis,dir(anIter)showsthatithasboth
__iter__()andnext()methods.
Becausethisobjectisaniterator,wecanuseaforstatementtoiterateoverthe
valuesreturnedbythegenerator.
Wecanalsogetitsvaluesbyrepeatedlycallingthenext()method,untilit
raisestheStopIterationexception.Thisabilitytocallthenextmethodenables
ustopasstheiteratorobjectaroundandgetvaluesatdifferentlocationsinour
code.
Oncewehaveobtainedallthevaluesfromaniterator,itis,ineffect,"empty"
or"exhausted".Theiteratorprotocol,infact,specifiesthatonceaniterator
raisestheStopIterationexception,itshouldcontinuetodoso.Anotherwayto
saythisisthatthereisno"rewind"operation.But,youcancallthethe
generatorfunctionagaintogeta"fresh"iterator.
Page57
2.3.2ExampleAclasscontainingageneratormethod
Eachtimethismethodiscalled,itproducesa(new)iteratorobject.Thismethodis
analogoustotheiterkeysanditervaluesmethodsinthedictionarybuiltinobject:
#
#Aclassthatprovidesaniteratorgeneratormethod.
#
classNode:
def__init__(self,name='<noname>',value='<novalue>',
children=None):
self.name=name
self.value=value
self.children=children
ifchildrenisNone:
self.children=[]
else:
defset_name(self,name):self.name=name
defget_name(self):returnself.name
defset_value(self,value):self.value=value
defget_value(self):returnself.value
defiterchildren(self):
forchildinself.children:
yieldchild
#
#Printinformationonthisnodeandwalkoverallchildren
and
#grandchildren...
defwalk(self,level=0):
print'%sname:%svalue:%s'%(
get_filler(level),self.get_name(),
self.get_value(),)
forchildinself.iterchildren():
child.walk(level+1)
#
#Anfunctionthatistheequivalentofthewalk()methodin
#classNode.
#
defwalk(node,level=0):
print'%sname:%svalue:%s'%(
get_filler(level),node.get_name(),node.get_value(),)
forchildinnode.iterchildren():
walk(child,level+1)
defget_filler(level):
return''*level
deftest():
a7=Node('gilbert','777')
a6=Node('fred','666')
a5=Node('ellie','555')
a4=Node('daniel','444')
a3=Node('carl','333',[a4,a5])
a2=Node('bill','222',[a6,a7])
a1=Node('alice','111',[a2,a3])
#Usethewalkmethodtowalktheentiretree.
print'Usingthemethod:'
Page58
a1.walk()
print'='*30
#Usethewalkfunctiontowalktheentiretree.
print'Usingthefunction:'
walk(a1)
test()
Usingthemethod:
name:alicevalue:111
name:billvalue:222
name:fredvalue:666
name:gilbertvalue:777
name:carlvalue:333
name:danielvalue:444
name:ellievalue:555
==============================
Usingthefunction:
name:alicevalue:111
name:billvalue:222
name:fredvalue:666
name:gilbertvalue:777
name:carlvalue:333
name:danielvalue:444
name:ellievalue:555
Thisclasscontainsamethoditerchildrenwhich,whencalled,returnsan
iterator.
Theyieldstatementinthemethoditerchildrenmakesitintoagenerator.
Theyieldstatementreturnsoneitemeachtimeitisreached.Thenexttimethe
iteratorobjectis"called"itresumesimmediatelyaftertheyieldstatement.
Afunctionmayhaveanynumberofyieldstatements.
Aforstatementwilliterateoveralltheitemsproducedbyaniteratorobject.
Thisexampleshowstwowaystousethegenerator,specifically:(1)thewalk
methodintheclassNodeand(2)thewalkfunction.Bothcallthegenerator
iterchildrenandbothdoprettymuchthesamething.
2.3.3ExampleAniteratorclass
Thisclassimplementstheiteratorprotocol.Therefore,instancesofthisclassare
iterators.Thepresenceofthenext()and__iter__()methodsmeansthatthisclass
implementstheiteratorprotocolandmakesinstancesofthisclassiterators.
Notethatwhenaniteratoris"exhausted"it,normally,cannotbereusedtoiterateover
thesequence.However,inthisexample,weprovidearefreshmethodwhichenables
usto"rewind"andreusetheiteratorinstance:
#
#Aniteratorclassthatdoes*not*use``yield``.
#Thisiteratorproduceseveryotheriteminasequence.
#
Page59
classIteratorExample:
def__init__(self,seq):
self.seq=seq
self.idx=0
defnext(self):
self.idx+=1
ifself.idx>=len(self.seq):
raiseStopIteration
value=self.seq[self.idx]
self.idx+=1
returnvalue
def__iter__(self):
returnself
defrefresh(self):
self.idx=0
deftest_iteratorexample():
a=IteratorExample('edcba')
forxina:
printx
print''
a.refresh()
forxina:
printx
print'='*30
a=IteratorExample('abcde')
try:
printa.next()
printa.next()
printa.next()
printa.next()
printa.next()
printa.next()
exceptStopIteration,e:
print'stopping',e
d
b
d
b
==============================
b
d
stopping
Thenextmethodmustkeeptrackofwhereitisandwhatitemitshould
producenext.
Alert:TheiteratorprotocolhaschangedslightlyinPython3.0.Inparticular,
thenext()methodhasbeenrenamedto__next__().See:PythonStandard
Library:IteratorTypes
http://docs.python.org/3.0/library/stdtypes.html#iteratortypes.
Page60
2.3.4ExampleAniteratorclassthatusesyield
Theremaybetimeswhenthenextmethodiseasierandmorestraightforwardto
implementusingyield.Ifso,thenthisclassmightserveasanmodel.Ifyoudonot
feeltheneedtodothis,thenyoushouldignorethisexample:
#
#Aniteratorclassthatuses``yield``.
#Thisiteratorproduceseveryotheriteminasequence.
#
classYieldIteratorExample:
def__init__(self,seq):
self.seq=seq
self.iterator=self._next()
self.next=self.iterator.next
def_next(self):
flag=0
forxinself.seq:
ifflag:
flag=0
yieldx
else:
flag=1
def__iter__(self):
returnself.iterator
defrefresh(self):
deftest_yielditeratorexample():
a=YieldIteratorExample('edcba')
forxina:
printx
print''
a.refresh()
forxina:
printx
print'='*30
a=YieldIteratorExample('abcde')
try:
printa.next()
printa.next()
printa.next()
printa.next()
printa.next()
printa.next()
exceptStopIteration,e:
print'stopping',e
test_yielditeratorexample()
d
b
d
b
==============================
Page61
b
d
stopping
Becausethe_nextmethodusesyield,callingit(actually,callingtheiterator
objectitproduces)inaniteratorcontextcausesittobe"resumed"immediately
aftertheyieldstatement.Thisreducesbookkeepingabit.
However,withthisstyle,wemustexplicitlyproduceaniterator.Wedothisby
callingthe_nextmethod,whichcontainsayieldstatement,andisthereforea
generator.Thefollowingcodeinourconstructor(__init__)completestheset
upofourclassasaniteratorclass:
Rememberthatweneedboth__iter__()andnext()methodsin
orderDictionarytosatisfytheiteratorprotocol.The__iter__()methodis
alreadythereandtheabovecodeintheconstructorcreatesthenext()method.
2.3.5ExampleAlistcomprehension
Alistcomprehensionlooksabitlikeaniterator,butitproducesalist.See:The
PythonLanguageReference:Listdisplays
http://docs.python.org/reference/expressions.html#listdisplaysformoreonlist
comprehensions.
Hereisanexample:
In[4]:deff(x):
...:returnx*3
...:
In[5]:list1=[11,22,33]
In[6]:list2=[f(x)forxinlist1]
In[7]:printlist2
[33,66,99]
2.3.6ExampleAgeneratorexpression
Ageneratorexpressionlooksquitesimilartoalistcomprehension,butisenclosedin
parenthesesratherthansquarebrackets.Unlikealistcomprehension,agenerator
expressiondoesnotproducealist;itproducesangeneratorobject.Ageneratorobject
isaniterator.
Formoreongeneratorexpressions,seeThePythonLanguageReference:Generator
expressionshttp://docs.python.org/reference/expressions.html#generator
expressions.
Thefollowingexampleusesageneratorexpressiontoproduceaniterator:
mylist=range(10)
deff(x):
Page62
returnx*3
genexpr=(f(x)forxinmylist)
forxingenexpr:
printx
Thegeneratorexpression(f(x)forxinmylist)producesaniteratorobject.
Noticethatwecanusetheiteratorobjectlaterinourcode,cansaveitinadata
structure,andcanpassittoafunction.
2.4UnitTests
UnittestandthePythonunittestframeworkprovideaconvenientwaytodefineand
runteststhatensurethataPythonapplicationproducesspecifiedresults.
Thissection,whileitwillnotattempttoexplaineverythingabouttheunittest
framework,willprovideexamplesofseveralstraightforwardwaystoconstructand
runtests.
Someassumptions:
Wearegoingtodevelopasoftwareprojectincrementally.Wewillnot
implementandreleaseallatonce.Therefore,eachtimeweaddtoourexisting
codebase,weneedawaytoverifythatouradditions(andfixes)havenot
causednewproblemsinoldcode.
Addingnewcodetoexistingcodewillcauseproblems.Weneedtobeableto
check/testforthoseproblemsateachstep.
Asweaddcode,weneedtobeabletoaddtestsforthatnewcode,too.
2.4.1Definingunittests
2.4.1.1Createatestclass.
Inthetestclass,implementanumberofmethodstoperformyourtests.Nameyour
testmethodswiththeprefix"test".Hereisanexample:
classMyTest:
deftest_one(self):
#sometestcode
pass
deftest_two(self):
#sometestcode
pass
Createatestharness.Hereisanexample:
#makethetestsuite.
defsuite():
loader=unittest.TestLoader()
testsuite=loader.loadTestsFromTestCase(MyTest)
returntestsuite
Page63
#Makethetestsuite;runthetests.
deftest():
testsuite=suite()
runner=unittest.TextTestRunner(sys.stdout,verbosity=2)
result=runner.run(testsuite)
Hereisamorecompleteexample:
importsys,StringIO,string
importunittest
importwebserv_example_heavy_sub
#Acomparisonfunctionforcaseinsenstivesorting.
defmycmpfunc(arg1,arg2):
returncmp(string.lower(arg1),string.lower(arg2))
classXmlTest(unittest.TestCase):
deftest_import_export1(self):
inFile=file('test1_in.xml','r')
inContent=inFile.read()
inFile.close()
doc=webserv_example_heavy_sub.parseString(inContent)
outFile=StringIO.StringIO()
outFile.write('<?xmlversion="1.0"?>\n')
doc.export(outFile,0)
outContent=outFile.getvalue()
outFile.close()
self.failUnless(inContent==outContent)
#makethetestsuite.
defsuite():
loader=unittest.TestLoader()
#Changethetestmethodprefix:test>trial.
#loader.testMethodPrefix='trial'
#Changethecomparisonfunctionthatdeterminestheorderof
tests.
#loader.sortTestMethodsUsing=mycmpfunc
testsuite=loader.loadTestsFromTestCase(XmlTest)
returntestsuite
#Makethetestsuite;runthetests.
deftest_main():
testsuite=suite()
runner=unittest.TextTestRunner(sys.stdout,verbosity=2)
result=runner.run(testsuite)
if__name__=="__main__":
test_main()
Runningtheabovescriptproducesthefollowingoutput:
test_import_export(__main__.XmlTest)...ok
Ran1testin0.035s
OK
Page64
Afewnotesonthisexample:
Thisexampleteststheabilitytoparseanxmldocumenttest1_in.xmland
exportthatdocumentbacktoXML.ThetestsucceedsiftheinputXML
documentandtheexportedXMLdocumentarethesame.
ThecodewhichisbeingtestedparsesanXMLdocumentreturnedbya
requesttoAmazonWebservices.YoucanlearnmoreaboutAmazonWeb
servicesat:http://www.amazon.com/webservices.Thiscodewasgenerated
fromanXMLSchemadocumentbygenerateDS.py.Soweareineffect,
testinggenerateDS.py.YoucanfindgenerateDS.pyat:
http://www.rexx.com/~dkuhlman/#generateDS.
Testingforsuccess/failureandreportingfailuresUsethemethodslistedat
http://www.python.org/doc/current/lib/testcaseobjects.htmltotestforand
reportsuccessandfailure.Inourexample,weused"self.failUnless(inContent
==outContent)"toensurethatthecontentweparsedandthecontentthatwe
exportedwerethesame.
Addadditionaltestsbyaddingmethodswhosenameshavetheprefix"test".If
youpreferadifferentprefixfortestsnames,addsomethinglikethefollowing
totheabovescript:
loader.testMethodPrefix='trial'
Bydefault,thetestsarerunintheorderoftheirnamessortedbythecmp
function.So,ifneeded,youcancontroltheorderofexecutionoftestsby
selectingtheirnames,forexample,usingnamesliketest_1_checkderef,
test_2_checkcalc,etc.Or,youcanchangethecomparisonfunctionbyadding
somethinglikethefollowingtotheabovescript:
loader.sortTestMethodsUsing=mycmpfunc
Asabitofmotivationforcreatingandusingunittests,whiledevelopingthisexample,
Idiscoveredseveralerrors(ormaybe"specialfeatures")ingenerateDS.py.
2.5ExtendingandembeddingPython
2.5.1Introductionandconcepts
Extendingvs.embeddingTheyaredifferentbutrelated:
ExtendingPythonmeanstoimplementanextensionmoduleoranextension
type.AnextensionmodulecreatesanewPythonmodulewhichis
implementedinC/C++.FromPythoncode,anextensionmoduleappearstobe
justlikeamoduleimplementedinPythoncode.Anextensiontypecreatesa
newPython(builtin)typewhichisimplementedinC/C++.FromPython
code,anextensiontypeappearstobejustlikeabuiltintype.
EmbeddingPython,bycontrast,istoputthePythoninterpreterwithinan
application(i.e.linkitin)sothattheapplicationcanrunPythonscripts.The
scriptscanbeexecutedortriggeredinavarietyofways,e.g.theycanbe
boundtokeysonthekeyboardortomenuitems,theycanbetriggeredby
Page65
externalevents,etc.Usually,inordertomaketheembeddedPython
interpreteruseful,Pythonisalsoextendedwithfunctionsfromtheembedding
application,sothatthescriptscancallfunctionsthatareimplementedbythe
embeddingC/C++application.
DocumentationThetwoimportantsourcesforinformationaboutextendingand
embeddingarethefollowing:
ExtendingandEmbeddingthePythonInterpreter
http://www.python.org/doc/current/ext/ext.html
Python/CAPIReferenceManual
http://www.python.org/doc/current/api/api.html
Typesofextensions:
ExtensionmodulesFromthePythonside,itappearstobeaPythonmodule.
Usuallyitexportsfunctions.
ExtensiontypesUsedtoimplementanewPythondatatype.
ExtensionclassesFromthePythonside,itappearstobeaclass.
ToolsThereareseveraltoolsthatsupportthedevelopmentofPythonextensions:
SWIGLearnaboutSWIGat:http://www.swig.org
PyrexLearnaboutPyrexat:
http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/
ThereisalsoCython,whichseemstobeanadvancedversionof,oratleastan
alternativetoPyrex.See:CythonCExtensionsforPython
http://www.cython.org/
2.5.2Extensionmodules
WritinganextensionmodulebyhandWhattodo:
Createthe"init"functionThenameofthisfunctionmustbe"init"followed
bythenameofthemodule.Everyextensionmodulemusthavesucha
function.
CreatethefunctiontableThistablemapsfunctionnames(referencedfrom
Pythoncode)tofunctionpointers(implementedinC/C++).
Implementeachwrapperfunction.
ImplementingawrapperfunctionWhattodo:
1. CapturetheargumentswithPyArg_ParseTuple.Theformatstringspecifies
howargumentsaretobeconvertedandcaptured.See1.7Extracting
ParametersinExtensionFunctions.Herearesomeofthemostcommonlyused
types:
Use"i","s","f",etctoconvertandcapturesimpletypessuchasintegers,
strings,floats,etc.
Use"O"togetapointertoPython"complex"typessuchaslists,tuples,
dictionaries,etc.
Useitemsinparenthesestocaptureandunpacksequences(e.g.listsand
tuples)offixedlength.Example:
if(!PyArg_ParseTuple(args,"(ii)(ii)",&x,&y,&width,
Page66
&height))
{
returnNULL;
}/*if*/
Asamplecallmightbe:
lowerLeft=(x1,y1)
extent=(width1,height1)
scan(lowerLeft,extent)
Use":aName"(colon)attheendoftheformatstringtoprovideafunction
nameforerrormessages.Example:
if(!PyArg_ParseTuple(args,"O:setContentHandler",
&pythonInstance))
{
returnNULL;
}/*if*/
Use";anerrormessage"(semicolon)attheendoftheformatstringto
provideastringthatreplacesthedefaulterrormessage.
Docsareavailableat:
http://www.python.org/doc/current/ext/parseTuple.html.
2. Writethelogic.
3. HandleerrorsandexceptionsYouwillneedtounderstandhowto(1)
clearingerrorsandexceptionsand(2)Raiseerrors(exceptions).
ManyfunctionsinthePythonCAPIraiseexceptions.Youwillneedto
checkforandcleartheseexceptions.Hereisanexample:
char*message;
intmessageNo;
message=NULL;
messageNo=1;
/*Istheargumentastring?
*/
if(!PyArg_ParseTuple(args,"s",&message))
{
/*It'snotastring.Cleartheerror.
*Thentrytogetamessagenumber(aninteger).
*/
PyErr_Clear();
if(!PyArg_ParseTuple(args,"i",&messageNo))
{
o
o
o
YoucanalsoraiseexceptionsinyourCcodethatcanbecaught(ina
"try:except:"block)backinthecallingPythoncode.Hereisanexample:
if(n==0)
{
PyErr_SetString(PyExc_ValueError,"Valuemustnotbe
zero");
returnNULL;
}
Page67
SeeInclude/pyerrors.hinthePythonsourcedistributionformore
exception/errortypes.
And,youcantestwhetherafunctioninthePythonCAPIthatyouhave
calledhasraisedanexception.Forexample:
if(PyErr_Occurred())
{
/*Anexceptionwasraised.
*Dosomethingaboutit.
*/
o
o
o
Formoredocumentationonerrorsandexceptions,see:
http://www.python.org/doc/current/api/exceptionHandling.html.
4. Createandreturnavalue:
ForeachbuiltinPythontypethereisasetofAPIfunctionstocreateand
manipulateit.Seethe"Python/CAPIReferenceManual"foradescription
ofthesefunctions.Forexample,see:
http://www.python.org/doc/current/api/intObjects.html
http://www.python.org/doc/current/api/stringObjects.html
http://www.python.org/doc/current/api/tupleObjects.html
http://www.python.org/doc/current/api/listObjects.html
http://www.python.org/doc/current/api/dictObjects.html
Etc.
ThereferencecountYouwillneedtofollowPython'srulesforreference
countingthatPythonusestogarbagecollectobjects.Youcanlearnabout
theserulesathttp://www.python.org/doc/current/ext/refcounts.html.You
willnotwantPythontogarbagecollectobjectsthatyoucreatetooearlyor
toolate.WithrespecttoPythonobjectscreatedwiththeabovefunctions,
thesenewobjectsareownedandmaybepassedbacktoPythoncode.
However,therearesituationswhereyourC/C++codewillnot
automaticallyownareference,forexamplewhenyouextractanobject
fromacontainer(alist,tuple,dictionary,etc).Inthesecasesyoushould
incrementthereferencecountwithPy_INCREF.
2.5.3SWIG
Note:OurdiscussionandexamplesareforSWIGversion1.3
SWIGwilloftenenableyoutogeneratewrappersforfunctionsinanexistingC
functionlibrary.SWIGdoesnotunderstandeverythinginCheaderfiles.Butitdoesa
fairlyimpressivejob.Youshouldtryitfirstbeforeresortingtothehardworkof
writingwrappersbyhand.
MoreinformationonSWIGisathttp://www.swig.org.
Herearesomestepsthatyoucanfollow:
1. CreateaninterfacefileEvenwhenyouarewrappingfunctionsdefinedinan
existingheaderfile,creatinganinterfacefileisagoodidea.Includeyour
Page68
existingheaderfileintoit,thenaddwhateverelseyouneed.Hereisan
extremelysimpleexampleofaSWIGinterfacefile:
%moduleMyLibrary
%{
#include"MyLibrary.h"
%}
%include"MyLibrary.h"
Comments:
The"%{"and"%}"bracketsaredirectivestoSWIG.Theysay:"Addthe
codebetweenthesebracketstothegeneratedwrapperfilewithout
processingit.
The"%include"statementsays:"Copythefileintotheinterfacefilehere.
Ineffect,youareaskingSWIGtogeneratewrappersforallthefunctions
inthisheaderfile.Ifyouwantwrappersforonlysomeofthefunctionsina
headerfile,thencopyorreproducefunctiondeclarationsforthedesired
functionshere.Anexample:
%moduleMyLibrary
%{
#include"MyLibrary.h"
%}
intcalcArea(intwidth,intheight);
intcalcVolume(intradius);
Thisexamplewillgeneratewrappersforonlytwofunctions.
YoucanfindmoreinformationaboutthedirectivesthatareusedinSWIG
interfacefilesintheSWIGUserManual,inparticularat:
http://www.swig.org/Doc1.3/Preprocessor.html
http://www.swig.org/Doc1.3/Python.html
2. Generatethewrappers:
swigpythonMyLibrary.i
3. Compileandlinkthelibrary.OnLinux,youcanusesomethinglikethe
following:
gcccMyLibrary.c
gcccI/usr/local/include/python2.3MyLibrary_wrap.c
gccsharedMyLibrary.oMyLibrary_wrap.oo_MyLibrary.so
Notethatweproduceasharedlibrarywhosenameisthemodulename
prefixedwithanunderscore.SWIGalsogeneratesa.pyfile,withoutthe
leadingunderscore,whichwewillimportfromourPythoncodeandwhich,in
turn,importsthesharedlibrary.
4. Usetheextensionmoduleinyourpythoncode:
Python2.3b1(#1,Apr252003,20:36:09)
[GCC2.95.420011002(Debianprerelease)]onlinux2
information.
Page69
>>>importMyLibrary
>>>MyLibrary.calcArea(4.0,5.0)
20.0
Hereisamakefilethatwillexecuteswigtogeneratewrappers,thencompileandlink
theextension.
CFLAGS=I/usr/local/include/python2.3
all:_MyLibrary.so
_MyLibrary.so:MyLibrary.oMyLibrary_wrap.o
MyLibrary.o:MyLibrary.c
gcccMyLibrary.coMyLibrary.o
MyLibrary_wrap.o:MyLibrary_wrap.c
gccc${CFLAGS}MyLibrary_wrap.coMyLibrary_wrap.o
MyLibrary_wrap.c:MyLibrary.i
clean:
rmfMyLibrary.pyMyLibrary.oMyLibrary_wrap.c
MyLibrary_wrap.o_MyLibrary.so
Hereisanexampleofrunningthismakefile:
$makefMyLibrary_makefileclean
rmfMyLibrary.pyMyLibrary.oMyLibrary_wrap.c\
MyLibrary_wrap.o_MyLibrary.so
$makefMyLibrary_makefile
gcccMyLibrary.coMyLibrary.o
gcccI/usr/local/include/python2.3MyLibrary_wrap.co
MyLibrary_wrap.o
And,hereareCsourcefilesthatcanbeusedinourexample.
MyLibrary.h:
/*MyLibrary.h
*/
floatcalcArea(floatwidth,floatheight);
floatcalcVolume(floatradius);
intgetVersion();
intgetMode();
MyLibrary.c:
/*MyLibrary.c
*/
Page70
floatcalcArea(floatwidth,floatheight)
{
return(width*height);
}
floatcalcVolume(floatradius)
{
return(3.14*radius*radius);
}
intgetVersion()
{
return123;
}
intgetMode()
{
return1;
}
2.5.4Pyrex
PyrexisausefultoolforwritingPythonextensions.BecausethePyrexlanguageis
similartoPython,writingextensionsinPyrexiseasierthandoingsoinC.Cython
appearstobetheanewerversionofPyrex.
MoreinformationisonPyrexandCythonisat:
Pyrexhttp://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/
CythonCExtensionsforPythonhttp://www.cython.org/
HereisasimplefunctiondefinitioninPyrex:
#python_201_pyrex_string.pyx
importstring
defformatString(objects1,objects2):
s1=string.strip(s1)
s2=string.strip(s2)
s3='<<%s||%s>>'%(s1,s2)
s4=s3*4
returns4
And,hereisamakefile:
CFLAGS=DNDEBUGO3WallWstrictprototypesfPIC\
I/usr/local/include/python2.3
all:python_201_pyrex_string.so
python_201_pyrex_string.so:python_201_pyrex_string.o
gccsharedpython_201_pyrex_string.oo
python_201_pyrex_string.so
python_201_pyrex_string.o:python_201_pyrex_string.c
gccc${CFLAGS}python_201_pyrex_string.co
Page71
python_201_pyrex_string.o
python_201_pyrex_string.c:python_201_pyrex_string.pyx
pyrexcpython_201_pyrex_string.pyx
clean:
rmfpython_201_pyrex_string.sopython_201_pyrex_string.o\
python_201_pyrex_string.c
Hereisanotherexample.Inthisone,onefunctioninthe.pyxfilecallsanother.Here
istheimplementationfile:
#python_201_pyrex_primes.pyx
defshowPrimes(intkmax):
plist=primes(kmax)
forpinplist:
print'prime:%d'%p
cdefprimes(intkmax):
cdefintn,k,i
cdefintp[1000]
result=[]
ifkmax>1000:
kmax=1000
k=0
n=2
whilek<kmax:
i=0
whilei<kandn%p[i]<>0:
i=i+1
ifi==k:
p[k]=n
k=k+1
result.append(n)
n=n+1
returnresult
#CFLAGS=DNDEBUGgO3WallWstrictprototypesfPIC#
I/usr/local/include/python2.3CFLAGS=DNDEBUGI/usr/local/
include/python2.3
all:python_201_pyrex_primes.so
python_201_pyrex_primes.so:python_201_pyrex_primes.o
gccsharedpython_201_pyrex_primes.oopython_201_pyrex_primes.so
python_201_pyrex_primes.o:python_201_pyrex_primes.c
gccc${CFLAGS}python_201_pyrex_primes.co
python_201_pyrex_primes.o
python_201_pyrex_primes.c:python_201_pyrex_primes.pyx
pyrexcpython_201_pyrex_primes.pyx
clean:
Page72
rmfpython_201_pyrex_primes.sopython_201_pyrex_primes.o
python_201_pyrex_primes.c
Hereistheoutputfromrunningthemakefile:
$makefpython_201_pyrex_makeprimesclean
rmfpython_201_pyrex_primes.sopython_201_pyrex_primes.o\
python_201_pyrex_primes.c
$makefpython_201_pyrex_makeprimes
pyrexcpython_201_pyrex_primes.pyx
gcccDNDEBUGI/usr/local/include/python2.3
python_201_pyrex_primes.copython_201_pyrex_primes.o
gccsharedpython_201_pyrex_primes.oo
python_201_pyrex_primes.so
Hereisaninteractiveexampleofitsuse:
$python
Python2.3b1(#1,Apr252003,20:36:09)
information.
>>>importpython_201_pyrex_primes
>>>dir(python_201_pyrex_primes)
['__builtins__','__doc__','__file__','__name__','showPrimes']
>>>python_201_pyrex_primes.showPrimes(5)
prime:2
prime:3
prime:5
prime:7
prime:11
ThisnextexampleshowshowtousePyrextoimplementanewextensiontype,thatis
anewPythonbuiltintype.Noticethattheclassisdeclaredwiththecdefkeyword,
whichtellsPyrextogeneratetheCimplementationofatypeinsteadofaclass.
Hereistheimplementationfile:
#python_201_pyrex_clsprimes.pyx
"""Animplementationofprimeshandlingclass
forademonstrationofPyrex.
"""
cdefclassPrimes:
"""Aclasscontainingfunctionsfor
handlingprimes.
"""
defshowPrimes(self,intkmax):
"""Showarangeofprimes.
Usethemethodprimes()togeneratetheprimes.
"""
plist=self.primes(kmax)
forpinplist:
print'prime:%d'%p
defprimes(self,intkmax):
"""Generatetheprimesintherange0kmax.
Page73
"""
cdefintn,k,i
cdefintp[1000]
result=[]
ifkmax>1000:
kmax=1000
k=0
n=2
whilek<kmax:
i=0
whilei<kandn%p[i]<>0:
i=i+1
ifi==k:
p[k]=n
k=k+1
result.append(n)
n=n+1
returnresult
CFLAGS=DNDEBUGI/usr/local/include/python2.3
all:python_201_pyrex_clsprimes.so
python_201_pyrex_clsprimes.so:python_201_pyrex_clsprimes.o
gccsharedpython_201_pyrex_clsprimes.oo
python_201_pyrex_clsprimes.so
python_201_pyrex_clsprimes.o:python_201_pyrex_clsprimes.c
gccc${CFLAGS}python_201_pyrex_clsprimes.co
python_201_pyrex_clsprimes.o
python_201_pyrex_clsprimes.c:python_201_pyrex_clsprimes.pyx
pyrexcpython_201_pyrex_clsprimes.pyx
clean:
rmfpython_201_pyrex_clsprimes.so
python_201_pyrex_clsprimes.o\
python_201_pyrex_clsprimes.c
Hereisoutputfromrunningthemakefile:
$makefpython_201_pyrex_makeclsprimesclean
rmfpython_201_pyrex_clsprimes.sopython_201_pyrex_clsprimes.o\
python_201_pyrex_clsprimes.c
$makefpython_201_pyrex_makeclsprimes
pyrexcpython_201_pyrex_clsprimes.pyx
gcccDNDEBUGI/usr/local/include/python2.3
python_201_pyrex_clsprimes.copython_201_pyrex_clsprimes.o
gccsharedpython_201_pyrex_clsprimes.oo
python_201_pyrex_clsprimes.so
Andhereisaninteractiveexampleofitsuse:
$python
Python2.3b1(#1,Apr252003,20:36:09)
information.
Page74
>>>importpython_201_pyrex_clsprimes
>>>dir(python_201_pyrex_clsprimes)
['Primes','__builtins__','__doc__','__file__','__name__']
>>>primes=python_201_pyrex_clsprimes.Primes()
>>>dir(primes)
['__class__','__delattr__','__doc__','__getattribute__',
'__hash__',
'__init__','__new__','__reduce__','__reduce_ex__','__repr__',
'__setattr__','__str__','primes','showPrimes']
>>>primes.showPrimes(4)
prime:2
prime:3
prime:5
prime:7
DocumentationAlsonoticethatPyrexpreservesthedocumentationforthemodule,
theclass,andthemethodsintheclass.Youcanshowthisdocumentationwithpydoc,
asfollows:
$pydocpython_201_pyrex_clsprimes
Or,inPythoninteractivemode,use:
$python
Python2.3b1(#1,Apr252003,20:36:09)
information.
>>>importpython_201_pyrex_clsprimes
>>>help(python_201_pyrex_clsprimes)
2.5.5SWIGvs.Pyrex
ChooseSWIGwhen:
YoualreadyhaveanexistingCorC++implementationofthecodeyouwant
tocallfromPython.InthiscaseyouwantSWIGtogeneratethewrappers.But
notethatCythonpromisestoenableyoutoquicklywrapandcallfunctions
implementedinC.
YouwanttowritetheimplementationinCorC++byhand.Perhaps,because
youthinkyoucandosoquickly,forexample,orbecauseyoubelievethatyou
canmakeithighlyoptimized.Then,youwanttobeabletogeneratethe
Python(extension)wrappersforitquickly.
ChoosePyrexwhen:
YoudonothaveaC/C++implementationandyouwantaneasierwaytowrite
thatCimplementation.WritingPyrexcode,whichisalotlikePython,is
easierthanwritingCorC++codebyhand).
YoustarttowritetheimplementationinC,thenfindthatitrequireslotsof
callstothePythonCAPI,andyouwanttoavoidhavingtolearnhowtodo
that.
Page75
2.5.6Cython
HereisasimpleexamplethatusesCythontowrapafunctionimplementedinC.
FirsttheCheaderfile:
/*test_c_lib.h*/
intcalculate(intwidth,intheight);
And,theCimplementationfile:
/*test_c_lib.c*/
#include"test_c_lib.h"
intcalculate(intwidth,intheight)
{
intresult;
result=width*height*3;
returnresult;
}
HereisaCythonfilethatcallsourCfunction:
#test_c.pyx
#DeclaretheexternalCfunction.
cdefexternfrom"test_c_lib.h":
intcalculate(intwidth,intheight)
deftest(w,h):
#CalltheexternalCfunction.
result=calculate(w,h)
print'resultfromcalculate:%d'%result
Wecancompileourcodeusingthisscript(onLinux):
#!/bin/bashx
cythontest_c.pyx
gcccfPICI/usr/local/include/python2.6otest_c.otest_c.c
gcccfPICI/usr/local/include/python2.6otest_c_lib.o
test_c_lib.c
gccsharedfPICI/usr/local/include/python2.6otest_c.so
test_c.otest_c_lib.o
HereisasmallPythonfilethatusesthewrapperthatwewroteinCython:
#run_test_c.py
importtest_c
deftest():
test_c.test(4,5)
test_c.test(12,15)
test()
And,whenwerunit,weseethefollowing:
Page76
$pythonrun_test_c.py
resultfromcalculate:60
resultfromcalculate:540
2.5.7Extensiontypes
ThegoalAnewbuiltindatatypeforPython.
ExistingexamplesObjects/listobject.c,Objects/stringobject.c,Objects/dictobject.c,
etcinthePythonsourcecodedistribution.
InolderversionsofthePythonsourcecodedistribution,atemplatefortheCcodewas
providedinObjects/xxobject.c.Objects/xxobject.cisnolongerincludedinthePython
sourcecodedistribution.However:
Thediscussionandexamplesforcreatingextensiontypeshavebeen
expanded.See:ExtendingandEmbeddingthePythonInterpreter,2.Defining
NewTypeshttp://docs.python.org/extending/newtypes.html.
IntheTools/framerdirectoryofthePythonsourcecodedistributionthereisan
applicationthatwillgenerateaskeletonforanextensiontypefroma
specificationobjectwritteninPython.RunTools/framer/example.pytoseeit
inaction.
And,youcanusePyrextogenerateanewbuiltintype.Todoso,implementa
Python/PyrexclassanddeclaretheclasswiththePyrexkeywordcdef.Infact,you
maywanttousePyrextogenerateaminimalextensiontype,andtheneditthat
generatedcodetoinsertandaddfunctionalitybyhand.SeethePyrexsectionforan
example.
Pyrexalsogoessomewaytowardgivingyouaccessto(existing)Cstructsand
functionsfromPython.
2.5.8Extensionclasses
ExtensionclassestheeasywaySWIGshadowclasses.
StartwithanimplementationofaC++classanditsheaderfile.
UsethefollowingSWIGflags:
swigc++pythonmymodule.i
MoreinformationisavailablewiththeSWIGdocumentationat:http://www.swig.org/
Doc1.3/Python.html.
ExtensionclassesthePyrexwayAnalternatieistousePyrextocompileaclass
definitionthatdoesnothavethecdefkeyword.UsingcdefontheclasstellsPyrexto
generateanextensiontypeinsteadofaclass.Youwillhavetodeterminewhetheryou
wantanextensionclassoranextensiontype.
Page77
2.6Parsing
Pythonisanexcellentlanguagefortextanalysis.
Insomecases,simplysplittinglinesoftextintowordswillbeenough.Inthesecases
usestring.split().
Inothercases,regularexpressionsmaybeabletodotheparsingyouneed.Ifso,see
thesectiononregularexpressionsinthisdocument.
However,insomecases,morecomplexanalysisofinputtextisrequired.Thissection
describessomeofthewaysthatPythoncanhelpyouwiththiscomplexparsingand
analysis.
2.6.1Specialpurposeparsers
ThereareanumberofspecialpurposeparserswhichyouwillfindinthePython
standardlibrary:
ConfigParserparserConfigurationfileparser
http://docs.python.org/library/configparser.html
getoptParserforcommandlineoptions
http://docs.python.org/library/getopt.html
optparseMorepowerfulcommandlineoptionparser
http://docs.python.org/library/optparse.html
urlparseParseURLsintocomponents
http://docs.python.org/library/urlparse.html
csvCSV(commaseparatedvalues)FileReadingandWriting
http://docs.python.org/library/csv.html#modulecsv
os.pathCommonpathnamemanipulations
http://docs.python.org/library/os.path.html
XMLparsersandXMLtoolsThereislotsofsupportforparsingandprocessing
XMLinPython.Hereareafewplacestolookforsupport:
ThePythonstandardlibraryStructuredMarkupProcessingTools
http://docs.python.org/library/markup.html.
Inparticular,youmaybeinterestedinxml.dom.minidomLightweightDOM
implementationhttp://docs.python.org/library/xml.dom.minidom.html.
ElementTreeYoucanthinkofElementTreeasanenhancedDOM
(documentobjectmodel).Manyfinditeasiertousethanminidom.
ElementTreeisinthePythonstandardlibrary,anddocumentationishere:
ElementTreeOverviewhttp://effbot.org/zone/elementindex.htm.
LxmlmimicstheElementTreeAPI,buthasadditionalcapabilities.Findout
aboutLxmlatlxmlhttp://codespeak.net/lxml/index.htmlNotethatlxml
alsohassupportforXPathandXSLT.
Dave'ssupportforPythonandXMLhttp://www.rexx.com/~dkuhlman.
2.6.2Writingarecursivedescentparserbyhand
Forsimplegrammars,thisisnotsohard.
Page78
Youwillneedtoimplement:
Arecognizermethodorfunctionforeachproductionruleinyourgrammar.
Eachrecognizermethodbeginslookingatthecurrenttoken,thenconsumesas
manytokensasneededtorecognizeit'sownproductionrule.Itcallsthe
recognizerfunctionsforanynonterminalsonitsrighthandside.
AtokenizerSomethingthatwillenableeachrecognizerfunctiontoget
tokens,onebyone.Thereareavarietyofwaystodothis,e.g.(1)afunction
thatproducesalistoftokensfromwhichrecognizerscanpoptokens;(2)a
generatorwhosenextmethodreturnsthenexttoken;etc.
Asanexample,we'llimplementarecursivedescentparserwritteninPythonforthe
followinggrammer:
Prog::=Command|CommandProg
Command::=Func_call
Func_call::=Term'('Func_call_list')'
Func_call_list::=Func_call|Func_call','Func_call_list
Term=<word>
Hereisanimplementationofarecursivedescentparserfortheabovegrammar:
#!/usr/bin/envpython
"""
Arecursivedescentparserexample.
Usage:
pythonrparser.py[options]<inputfile>
Options:
h,helpDisplaythishelpmessage.
Example:
pythonrparser.pymyfile.txt
Thegrammar:
Command::=Func_call
Term=<word>
"""
importsys
importstring
importtypes
importgetopt
#
#TousetheIPythoninteractiveshelltoinspectyourrunning
#application,uncommentthefollowinglines:
#
##fromIPython.ShellimportIPShellEmbed
##ipshell=IPShellEmbed((),
##banner='>>>>>>>>IntoIPython>>>>>>>>',
##exit_msg='<<<<<<<<OutofIPython<<<<<<<<')
#
#Thenaddthefollowinglineatthepointinyourcodewhere
#youwanttoinspectruntimevalues:
Page79
#
#ipshell('somemessagetoidentifywhereweare')
#
#Formoreinformationsee:http://ipython.scipy.org/moin/
#
#
#Constants
#
#ASTnodetypes
NoneNodeType=0
ProgNodeType=1
CommandNodeType=2
FuncCallNodeType=3
FuncCallListNodeType=4
TermNodeType=5
#Tokentypes
NoneTokType=0
LParTokType=1
RParTokType=2
WordTokType=3
CommaTokType=4
EOFTokType=5
#Dictionarytomapnodetypevaluestonodetypenames
NodeTypeDict={
NoneNodeType:'NoneNodeType',
ProgNodeType:'ProgNodeType',
CommandNodeType:'CommandNodeType',
FuncCallNodeType:'FuncCallNodeType',
FuncCallListNodeType:'FuncCallListNodeType',
TermNodeType:'TermNodeType',
}
#
#RepresentationofanodeintheAST(abstractsyntaxtree).
#
classASTNode:
def__init__(self,nodeType,*args):
self.nodeType=nodeType
self.children=[]
foriteminargs:
self.children.append(item)
defshow(self,level):
self.showLevel(level)
print'NodeType%s'%NodeTypeDict[self.nodeType]
level+=1
ifisinstance(child,ASTNode):
child.show(level)
eliftype(child)==types.ListType:
foriteminchild:
item.show(level)
else:
print'Child:',child
defshowLevel(self,level):
foridxinrange(level):
Page80
print'',
#
#Therecursivedescentparserclass.
#Containsthe"recognizer"methods,whichimplementthegrammar
#rules(above),onerecognizermethodforeachproductionrule.
#
classProgParser:
def__init__(self):
pass
defparseFile(self,infileName):
self.infileName=infileName
self.tokens=None
self.tokenType=NoneTokType
self.token=''
self.lineNo=1
self.infile=file(self.infileName,'r')
self.tokens=genTokens(self.infile)
try:
self.tokenType,self.token,self.lineNo=
self.tokens.next()
exceptStopIteration:
raiseRuntimeError,'Emptyfile'
result=self.prog_reco()
self.infile.close()
self.infile=None
returnresult
defparseStream(self,instream):
self.tokens=genTokens(instream,'<instream>')
try:
self.tokens.next()
returnresult
defprog_reco(self):
commandList=[]
while1:
result=self.command_reco()
ifnotresult:
break
commandList.append(result)
returnASTNode(ProgNodeType,commandList)
defcommand_reco(self):
ifself.tokenType==EOFTokType:
returnNone
result=self.func_call_reco()
returnASTNode(CommandNodeType,result)
deffunc_call_reco(self):
ifself.tokenType==WordTokType:
term=ASTNode(TermNodeType,self.token)
self.tokens.next()
ifself.tokenType==LParTokType:
Page81
self.tokens.next()
result=self.func_call_list_reco()
ifresult:
ifself.tokenType==RParTokType:
self.tokenType,self.token,self.lineNo
=\
self.tokens.next()
returnASTNode(FuncCallNodeType,term,
result)
else:
raiseParseError(self.lineNo,'missing
rightparen')
else:
raiseParseError(self.lineNo,'badfunccall
list')
else:
raiseParseError(self.lineNo,'missingleft
paren')
else:
returnNone
deffunc_call_list_reco(self):
terms=[]
while1:
ifnotresult:
break
terms.append(result)
ifself.tokenType!=CommaTokType:
break
self.tokens.next()
returnASTNode(FuncCallListNodeType,terms)
#
#Theparseerrorexceptionclass.
#
classParseError(Exception):
def__init__(self,lineNo,msg):
RuntimeError.__init__(self,msg)
self.lineNo=lineNo
self.msg=msg
defgetLineNo(self):
returnself.lineNo
defgetMsg(self):
returnself.msg
defis_word(token):
forletterintoken:
ifletternotinstring.ascii_letters:
returnNone
return1
#
#Generatethetokens.
#Usage:
#gen=genTokens(infile)
#tokType,tok,lineNo=gen.next()
#...
Page82
defgenTokens(infile):
lineNo=0
while1:
lineNo+=1
try:
line=infile.next()
except:
yield(EOFTokType,None,lineNo)
toks=line.split()
fortokintoks:
ifis_word(tok):
tokType=WordTokType
eliftok=='(':
tokType=LParTokType
eliftok==')':
tokType=RParTokType
eliftok==',':
tokType=CommaTokType
yield(tokType,tok,lineNo)
deftest(infileName):
parser=ProgParser()
#ipshell('(test)#1\nCtrlDtoexit')
result=None
try:
result=parser.parseFile(infileName)
exceptParseError,exp:
sys.stderr.write('ParseError:(%d)%s\n'%\
(exp.getLineNo(),exp.getMsg()))
ifresult:
result.show(0)
defusage():
print__doc__
sys.exit(1)
defmain():
args=sys.argv[1:]
try:
opts,args=getopt.getopt(args,'h',['help'])
except:
usage()
relink=1
foropt,valinopts:
ifoptin('h','help'):
usage()
iflen(args)!=1:
usage()
inputfile=args[0]
test(inputfile)
#importpdb;pdb.set_trace()
main()
Commentsandexplanation:
ThetokenizerisaPythongenerator.ItreturnsaPythongeneratorthatcan
produce"(tokType,tok,lineNo)"tuples.Ourtokenizerissosimpleminded
thatwehavetoseparateallofourtokenswithwhitespace.(Alittlelater,we'll
Page83
seehowtousePlextoovercomethislimitation.)
Theparserclass(ProgParser)containstherecognizermethodsthatimplement
theproductionrules.Eachofthesemethodsrecognizesasyntacticconstruct
definedbyarule.Inourexample,thesemethodshavenamesthatendwith
"_reco".
Wecouldhave,alternatively,implementedourrecognizersasglobal
functions,insteadofasmethodsinaclass.However,usingaclassgivesusa
placeto"hang"thevariablesthatareneededacrossmethodsandsavesus
fromhavingtouse("evil")globalvariables.
Arecognizermethodrecognizesaterminals(syntacticelementsontheright
handsideofthegrammarruleforwhichthereisnogrammarrule)by(1)
checkingthetokentypeandthetokenvalue,andthen(2)callingthetokenizer
togetthenexttoken(becauseithasconsumedatoken).
Arecognizermethodchecksforandprocessesanonterminal(syntactic
elementsontherighthandsideforwhichthereisagrammarrule)bycalling
therecognizermethodthatimplementsthatnonterminal.
Ifarecognizermethodfindsasyntaxerror,itraisesanexceptionofclass
ParserError.
SinceourexamplerecursivedescentparsercreatesanAST(anabstractsyntax
tree),wheneverarecognizermethodsuccessfullyrecognizesasyntactic
construct,itcreatesaninstanceofclassASTNodetorepresentitandreturns
thatinstancetoitscaller.TheinstanceofASTNodehasanodetypeand
containschildnodeswhichwereconstructedbyrecognizermethodscalledby
thisone(i.e.thatrepresentnonterminalsontherighthandsideofagrammar
rule).
Eachtimearecognizermethod"consumesatoken",itcallsthetokenizerto
getthenexttoken(andtokentypeandlinenumber).
Thetokenizerreturnsatokentypeinadditiontothetokenvalue.Italso
returnsalinenumberforerrorreporting.
ThesyntaxtreeisconstructedfrominstancesofclassASTNode.
TheASTNodeclasshasashowmethod,whichwalkstheASTandproduces
output.Youcanimaginethatasimilarmethodcoulddocodegeneration.And,
youshouldconsiderthepossibilityofwritinganalogoustreewalkmethods
thatperformtaskssuchasoptimization,annotationoftheAST,etc.
And,hereisasampleofthedatawecanapplythisparserto:
aaa()
bbb(ccc())
ddd(eee(),fff(ggg(),hhh(),iii()))
And,ifweruntheparseronthethisinputdata,wesee:
$pythonworkbook045.pyworkbook045.data
NodeTypeProgNodeType
NodeTypeCommandNodeType
NodeTypeFuncCallNodeType
NodeTypeTermNodeType
Child:aaa
NodeTypeFuncCallListNodeType
Page84
Child:bbb
Child:ccc
Child:ddd
Child:eee
Child:fff
Child:ggg
Child:hhh
Child:iii
2.6.3Creatingalexer/tokenizerwithPlex
LexicalanalysisThetokenizerinourrecursivedescentparserexamplewas(for
demonstrationpurposes)overlysimple.Youcanalwayswritemorecomplex
tokenizersbyhand.However,formorecomplex(andreal)tokenizers,youmaywant
touseatooltobuildyourtokenizer.
Inthissectionwe'lldescribePlexanduseittoproduceatokenizerforourrecursive
descentparser.
YoucanobtainPlexathttp://www.cosc.canterbury.ac.nz/~greg/python/Plex/.
Inordertouseit,youmaywanttoaddPlex1.1.4/PlextoyourPYTHONPATH.
HereisasimpleexamplefromthePlextutorial:
"""
SamplePlexlexer
Usage:
pythonplex_example.pyinputfile
Page85
"""
importsys
importPlex
defcount_lines(scanner,text):
scanner.line_count+=1
print''*60
letter=Plex.Range("AZaz")
digit=Plex.Range("09")
name=letter+Plex.Rep(letter|digit)
number=Plex.Rep1(digit)
space=Plex.Any("\t")
endline=Plex.Str('\n')
#comment=Plex.Str('"')+Plex.Rep(Plex.AnyBut('"'))+
Plex.Str('"')
resword=Plex.Str("if","then","else","end")
lexicon=Plex.Lexicon([
(endline,count_lines),
(resword,'keyword'),
(name,'ident'),
(number,'int'),
(Plex.Any("+*/=<>"),'operator'),
(space,Plex.IGNORE),
#(comment,'comment'),
(Plex.Str('('),'lpar'),
(Plex.Str(')'),'rpar'),
#commentssurroundedby(*and*)
(Plex.Str("(*"),Plex.Begin('comment')),
Plex.State('comment',[
(Plex.Str("*)"),Plex.Begin('')),
(Plex.AnyChar,Plex.IGNORE),
]),
])
infile=open(infileName,"r")
scanner=Plex.Scanner(lexicon,infile,infileName)
scanner.line_count=0
whileTrue:
token=scanner.read()
iftoken[0]isNone:
break
position=scanner.position()
posstr=('(%d,%d)'%(position[1],
position[2],)).ljust(10)
tokstr='"%s"'%token[1]
tokstr=tokstr.ljust(20)
print'%stok:%stokType:%s'%(posstr,tokstr,
token[0],)
print'line_count:%d'%scanner.line_count
defusage():
print__doc__
sys.exit(1)
defmain():
args=sys.argv[1:]
iflen(args)!=1:
Page86
usage()
infileName=args[0]
test(infileName)
main()
Hereisabitofdataonwhichwecanusetheabovelexer:
mass=(height*(*somecomment*)width*depth)/density
totalmass=totalmass+mass
And,whenweapplytheabovetestprogramtothisdata,hereiswhatwesee:
$pythonplex_example.pyplex_example.data
(1,0)tok:"mass"tokType:ident
(1,5)tok:"="tokType:operator
(1,7)tok:"("tokType:lpar
(1,8)tok:"height"tokType:ident
(1,15)tok:"*"tokType:operator
(1,36)tok:"width"tokType:ident
(1,42)tok:"*"tokType:operator
(1,44)tok:"depth"tokType:ident
(1,49)tok:")"tokType:rpar
(1,51)tok:"/"tokType:operator
(1,53)tok:"density"tokType:ident
(2,0)tok:"totalmass"tokType:ident
(2,10)tok:"="tokType:operator
(2,12)tok:"totalmass"tokType:ident
(2,22)tok:"+"tokType:operator
(2,24)tok:"mass"tokType:ident
line_count:2
Createalexiconfromscanningpatterns.
SeethePlextutorialandreference(andbelow)formoreinformationonhow
toconstructthepatternsthatmatchvarioustokens.
Createascannerwithalexicon,aninputfile,andaninputfilename.
Thecall"scanner.read()"getsthenexttoken.Itreturnsatuplecontaining(1)
thetokenvalueand(2)thetokentype.
Thecall"scanner.position()"getsthepositionofthecurrenttoken.Itreturnsa
tuplecontaining(1)theinputfilename,(2)thelinenumber,and(3)the
columnnumber.
Wecanexecuteamethodwhenagiventokenisfoundbyspecifyingthe
functionasthetokenaction.Inourexample,thefunctioniscount_lines.
Maintainingalinecountisactuallyunneeded,sincethepositiongivesusthis
information.However,noticehowweareabletomaintainavalue(inourcase
line_count)asanattributeofthescanner.
And,herearesomecommentsonconstructingthepatternsusedinalexicon:
Plex.Rangeconstructsapatternthatmatchesanycharacterintherange.
Page87
Plex.Repconstructsapatternthatmatchesasequenceofzeroormoreitems.
Plex.Rep1constructsapatternthatmatchesasequenceofoneormoreitems.
pat1 + pat2constructsapatternthatmatchesasequencecontainingpat1
followedbypat2.
pat1 | pat2constructsapatternthatmatcheseitherpat1orpat2.
Plex.Anyconstructsapatternthatmatchesanyonecharacterinitsargument.
Nowlet'srevisitourrecursivedescentparser,thistimewithatokenizerbuiltwith
Plex.Thetokenizeristrivial,butwillserveasanexampleofhowtohookitintoa
parser:
"""
ArecursivedescentparserexampleusingPlex.
ThisexampleusesPlextoimplementatokenizer.
Usage:
pythonpython_201_rparser_plex.py[options]<inputfile>
Options:
Example:
pythonpython_201_rparser_plex.pymyfile.txt
Thegrammar:
Command::=Func_call
Term=<word>
"""
importsys,string,types
importgetopt
importPlex
##fromIPython.ShellimportIPShellEmbed
##ipshell=IPShellEmbed((),
##banner='>>>>>>>>IntoIPython>>>>>>>>',
##exit_msg='<<<<<<<<OutofIPython<<<<<<<<')
#
#Constants
#
#ASTnodetypes
NoneNodeType=0
ProgNodeType=1
CommandNodeType=2
FuncCallNodeType=3
TermNodeType=5
#Tokentypes
NoneTokType=0
LParTokType=1
Page88
RParTokType=2
WordTokType=3
CommaTokType=4
EOFTokType=5
NodeTypeDict={
}
#
#
classASTNode:
self.children=[]
foriteminargs:
print'NodeType%s'%NodeTypeDict[self.nodeType]
level+=1
child.show(level)
foriteminchild:
item.show(level)
else:
print'Child:',child
print'',
#
#Therecursivedescentparserclass.
#Containsthe"recognizer"methods,whichimplementthegrammar
#rules(above),onerecognizermethodforeachproductionrule.
#
classProgParser:
def__init__(self):
self.tokens=None
self.token=''
self.lineNo=1
self.infile=None
self.tokens=None
defparseFile(self,infileName):
self.tokens=None
self.token=''
self.lineNo=1
Page89
self.infile=file(infileName,'r')
self.tokens=genTokens(self.infile,infileName)
try:
self.tokens.next()
self.infile.close()
self.infile=None
returnresult
defparseStream(self,instream):
self.tokens=None
self.token=''
self.lineNo=1
self.tokens=genTokens(self.instream,'<stream>')
try:
self.tokens.next()
raiseRuntimeError,'Emptystream'
self.infile.close()
self.infile=None
returnresult
defprog_reco(self):
commandList=[]
while1:
result=self.command_reco()
ifnotresult:
break
commandList.append(result)
returnASTNode(ProgNodeType,commandList)
defcommand_reco(self):
ifself.tokenType==EOFTokType:
returnNone
returnASTNode(CommandNodeType,result)
deffunc_call_reco(self):
ifself.tokenType==WordTokType:
term=ASTNode(TermNodeType,self.token)
self.tokens.next()
ifself.tokenType==LParTokType:
self.tokens.next()
result=self.func_call_list_reco()
ifresult:
ifself.tokenType==RParTokType:
self.tokenType,self.token,self.lineNo
=\
self.tokens.next()
returnASTNode(FuncCallNodeType,term,
result)
else:
Page90
raiseParseError(self.lineNo,'missing
rightparen')
else:
raiseParseError(self.lineNo,'badfunccall
list')
else:
raiseParseError(self.lineNo,'missingleft
paren')
else:
returnNone
deffunc_call_list_reco(self):
terms=[]
while1:
ifnotresult:
break
terms.append(result)
ifself.tokenType!=CommaTokType:
break
self.tokens.next()
returnASTNode(FuncCallListNodeType,terms)
#
#Theparseerrorexceptionclass.
#
classParseError(Exception):
def__init__(self,lineNo,msg):
RuntimeError.__init__(self,msg)
self.lineNo=lineNo
self.msg=msg
defgetLineNo(self):
returnself.lineNo
defgetMsg(self):
returnself.msg
#
#Generatethetokens.
#Usageexample
#gen=genTokens(infile)
#tokType,tok,lineNo=gen.next()
#...
defgenTokens(infile,infileName):
letter=Plex.Range("AZaz")
digit=Plex.Range("09")
name=letter+Plex.Rep(letter|digit)
lpar=Plex.Str('(')
rpar=Plex.Str(')')
comma=Plex.Str(',')
comment=Plex.Str("#")+Plex.Rep(Plex.AnyBut("\n"))
space=Plex.Any("\t\n")
lexicon=Plex.Lexicon([
(name,'word'),
(lpar,'lpar'),
(rpar,'rpar'),
(comma,'comma'),
(comment,Plex.IGNORE),
(space,Plex.IGNORE),
])
Page91
scanner=Plex.Scanner(lexicon,infile,infileName)
while1:
tokenType,token=scanner.read()
name,lineNo,columnNo=scanner.position()
iftokenType==None:
tokType=EOFTokType
token=None
eliftokenType=='word':
tokType=WordTokType
eliftokenType=='lpar':
tokType=LParTokType
eliftokenType=='rpar':
tokType=RParTokType
eliftokenType=='comma':
tokType=CommaTokType
else:
tokType=NoneTokType
tok=token
yield(tokType,tok,lineNo)
parser=ProgParser()
#ipshell('(test)#1\nCtrlDtoexit')
result=None
try:
result=parser.parseFile(infileName)
exceptParseError,exp:
sys.stderr.write('ParseError:(%d)%s\n'%\
(exp.getLineNo(),exp.getMsg()))
ifresult:
result.show(0)
defusage():
print__doc__
sys.exit(1)
defmain():
args=sys.argv[1:]
try:
except:
usage()
foropt,valinopts:
usage()
iflen(args)!=1:
usage()
infileName=args[0]
test(infileName)
main()
And,hereisasampleofthedatawecanapplythisparserto:
#TestforrecursivedescentparserandPlex.
#Command#1
aaa()
#Command#2
Page92
bbb(ccc())#Anendoflinecomment.
#Command#3
#Endoftest
And,whenwerunourparser,itproducesthefollowing:
$pythonplex_recusive.pyplex_recusive.data
NodeTypeProgNodeType
Child:aaa
Child:bbb
Child:ccc
Child:ddd
Child:eee
Child:fff
Child:ggg
Child:hhh
Child:iii
Comments:
Wecannowputcommentsinourinput,andtheywillbeignored.Comments
beginwitha"#"andcontinuetotheendofline.Seethedefinitionofcomment
infunctiongenTokens.
Thistokenizerdoesnotrequireustoseparatetokenswithwhitespaceasdid
thesimpletokenizerintheearlierversionofourrecursivedescentparser.
Thechangeswemadeovertheearlierversionwereto:
1. ImportPlex.
Page93
2. ReplacethedefinitionofthetokenizerfunctiongenTokens.
3. ChangethecalltogenTokenssothatthecallpassesinthefilename,
whichisneededtocreatethescanner.
OurnewversionofgenTokensdoesthefollowing:
4. Createpatternsforscanning.
5. Createalexicon(aninstanceofPlex.Lexicon),whichusesthepatterns.
6. Createascanner(aninstanceofPlex.Scanner),whichusesthelexicon.
7. Executealoopthatreadstokens(fromthescanner)and"yields"eachone.
2.6.4Asurveyofexistingtools
Forcomplexparsingtasks,youmaywanttoconsiderthefollowingtools:
kwParsingAparsergeneratorinPython
http://gadfly.sourceforge.net/kwParsing.html
PLYPythonLexYacchttp://systems.cs.uchicago.edu/ply/
PyLRFastLRparsinginpython
http://starship.python.net/crew/scott/PyLR.html
YappsTheYappsParserGeneratorSystem
http://theory.stanford.edu/~amitp/Yapps/
And,forlexicalanalysis,youmayalsowanttolookhere:
UsingRegularExpressionsforLexicalAnalysishttp://effbot.org/zone/xml
scanner.htm
Plexhttp://www.cosc.canterbury.ac.nz/~greg/python/Plex/.
Inthesectionsbelow,wegiveexamplesandnotesabouttheuseofPLYand
pyparsing.
2.6.5CreatingaparserwithPLY
InthissectionwewillshowhowtoimplementourparserexamplewithPLY.
FirstdownloadPLY.Itisavailablehere:PLY(PythonLexYacc)
http://www.dabeaz.com/ply/
ThenaddthePLYdirectorytoyourPYTHONPATH.
LearnhowtoconstructlexersandparserswithPLYbyreadingdoc/ply.htmlinthe
distributionofPLYandbylookingattheexamplesinthedistribution.
Forthoseofyouwhowantamorecomplexexample,seeAPythonParserforthe
RELAXNGCompactSyntax,whichisimplementedwithPLY.
Now,hereisourexampleparser.Commentsandexplanationsarebelow:
"""
Aparserexample.
ThisexampleusesPLYtoimplementalexerandparser.
Thegrammar:
Page94
Prog::=Command*
Command::=Func_call
Func_call_list::=Func_call*
Term=<word>
Hereisasample"program"touseasinput:
#Command#1
aaa()
#Command#2
#Command#3
#Endoftest
"""
importsys
importtypes
importgetopt
importply.lexaslex
importply.yaccasyacc
#
#Globals
#
startlinepos=0
#
#Constants
#
#ASTnodetypes
NoneNodeType=0
ProgNodeType=1
CommandNodeType=2
CommandListNodeType=3
FuncCallNodeType=4
TermNodeType=6
NodeTypeDict={
CommandListNodeType:'CommandListNodeType',
}
#
#
classASTNode:
Page95
self.children=[]
foriteminargs:
defappend(self,item):
print'NodeType:%s'%NodeTypeDict[self.nodeType]
level+=1
child.show(level)
foriteminchild:
item.show(level)
else:
print'Value:',child
print'',
#
#Exceptionclasses
#
classLexerError(Exception):
def__init__(self,msg,lineno,columnno):
self.msg=msg
self.lineno=lineno
self.columnno=columnno
defshow(self):
sys.stderr.write('Lexererror(%d,%d)%s\n'%\
(self.lineno,self.columnno,self.msg))
classParserError(Exception):
def__init__(self,msg,lineno,columnno):
self.msg=msg
self.lineno=lineno
self.columnno=columnno
defshow(self):
sys.stderr.write('Parsererror(%d,%d)%s\n'%\
(self.lineno,self.columnno,self.msg))
#
#Lexerspecification
#
tokens=(
'NAME',
'LPAR','RPAR',
'COMMA',
)
#Tokens
t_LPAR=r'$'
t_RPAR=r'$'
t_COMMA=r'\,'
t_NAME=r'[azAZ_][azAZ09_]*'
Page96
#Ignorewhitespace
t_ignore='\t'
#Ignorecomments('#'toendofline)
deft_COMMENT(t):
r'\#[^\n]*'
pass
deft_newline(t):
r'\n+'
globalstartlinepos
startlinepos=t.lexer.lexpos1
t.lineno+=t.value.count("\n")
deft_error(t):
globalstartlinepos
msg="Illegalcharacter'%s'"%(t.value[0])
columnno=t.lexer.lexposstartlinepos
raiseLexerError(msg,t.lineno,columnno)
#
#Parserspecification
#
defp_prog(t):
'prog:command_list'
t[0]=ASTNode(ProgNodeType,t[1])
defp_command_list_1(t):
'command_list:command'
t[0]=ASTNode(CommandListNodeType,t[1])
defp_command_list_2(t):
'command_list:command_listcommand'
t[1].append(t[2])
t[0]=t[1]
defp_command(t):
'command:func_call'
t[0]=ASTNode(CommandNodeType,t[1])
defp_func_call_1(t):
'func_call:termLPARRPAR'
t[0]=ASTNode(FuncCallNodeType,t[1])
defp_func_call_2(t):
'func_call:termLPARfunc_call_listRPAR'
t[0]=ASTNode(FuncCallNodeType,t[1],t[3])
defp_func_call_list_1(t):
'func_call_list:func_call'
t[0]=ASTNode(FuncCallListNodeType,t[1])
defp_func_call_list_2(t):
'func_call_list:func_call_listCOMMAfunc_call'
t[1].append(t[3])
t[0]=t[1]
defp_term(t):
'term:NAME'
t[0]=ASTNode(TermNodeType,t[1])
Page97
defp_error(t):
globalstartlinepos
msg="Syntaxerrorat'%s'"%t.value
columnno=t.lexer.lexposstartlinepos
raiseParserError(msg,t.lineno,columnno)
#
#ParsetheinputanddisplaytheAST(abstractsyntaxtree)
#
defparse(infileName):
startlinepos=0
#Buildthelexer
lex.lex(debug=1)
#Buildtheparser
yacc.yacc()
#Readtheinput
infile=file(infileName,'r')
content=infile.read()
infile.close()
try:
#Dotheparse
result=yacc.parse(content)
#DisplaytheAST
result.show(0)
exceptLexerError,exp:
exp.show()
exceptParserError,exp:
exp.show()
USAGE_TEXT=__doc__
defusage():
printUSAGE_TEXT
sys.exit(1)
defmain():
args=sys.argv[1:]
try:
except:
usage()
relink=1
foropt,valinopts:
usage()
iflen(args)!=1:
usage()
infileName=args[0]
parse(infileName)
main()
Applyingthisparsertothefollowinginput:
#Command#1
aaa()
Page98
#Command#2
#Command#3
#Endoftest
producesthefollowingoutput:
NodeType:ProgNodeType
NodeType:CommandListNodeType
NodeType:CommandNodeType
NodeType:FuncCallNodeType
NodeType:TermNodeType
Value:aaa
Value:bbb
NodeType:FuncCallListNodeType
Value:ccc
Value:ddd
Value:eee
Value:fff
Value:ggg
Value:hhh
Value:iii
CreatingthesyntaxtreeBasically,eachrule(1)recognizesanonterminal,
(2)createsanode(possiblyusingthevaluesfromtherighthandsideofthe
rule),and(3)returnsthenodebysettingthevalueoft[0].Adeviationfrom
thisistheprocessingofsequences,discussedbelow.
Sequencesp_command_list_1andp_command_list_1showhowtohandle
sequencesofitems.Inthiscase:
p_command_list_1recognizesacommandandcreatesaninstanceof
ASTNodewithtypeCommandListNodeTypeandaddsthecommandtoit
asachild,and
p_command_list_2recognizesanadditionalcommandandaddsit(asa
child)totheinstanceofASTNodethatrepresentsthelist.
Page99
DistinguishingbetweendifferentformsofthesameruleInordertoprocess
alternativestothesameproductionruledifferently,weusedifferentfunctions
withdifferentimplementations.Forexample,weuse:
p_func_call_1torecognizeandprocess"func_call:termLPARRPAR"(a
functioncallwithoutarguments),and
p_func_call_2torecognizeandprocess"func_call:termLPAR
func_call_listRPAR"(afunctioncallwitharguments).
ReportingerrorsOurparserreportsthefirsterrorandquits.We'vedonethis
byraisinganexceptionwhenwefindanerror.Weimplementtwoexception
classes:LexerErrorandParserError.Implementingmorethanoneexception
classenablesustodistinguishbetweendifferentclassesoferrors(notethe
multipleexcept:clausesonthetry:statementinfunctionparse).And,weuse
aninstanceoftheexceptionclassasacontainerinorderto"bubbleup"
informationabouttheerror(e.g.amessage,alinenumber,andacolumn
number).
2.6.6Creatingaparserwithpyparsing
pyparsingisarelativelynewparsingpackageforPython.Itwasimplementedandis
supportedbyPaulMcGuireanditshowspromise.Itappearsespeciallyeasytouse
andseemsespeciallyappropriateinparticularforquickparsingtasks,althoughithas
featuresthatmakesomecomplexparsingtaskseasy.ItfollowsaverynaturalPython
styleforconstructingparsers.
Gooddocumentationcomeswiththepyparsingdistribution.Seefile
HowToUseParsing.html.So,Iwon'ttrytorepeatthathere.Whatfollowsisanattempt
toprovideseveralquickexamplestohelpyousolvesimpleparsingtasksasquicklyas
possible.
Youwillalsowanttolookatthesamplesintheexamplesdirectory,whicharevery
helpful.Myexamplesbelowarefairlysimple.Youcanseemoreoftheabilityof
pyparsingtohandlecomplextasksintheexamples.
WheretogetitYoucanfindpyparsingat:PyparsingWikiHome
http://pyparsing.wikispaces.com/
HowtoinstallitPutthepyparsingmodulesomewhereonyourPYTHONPATH.
Andnow,hereareafewexamples.
2.6.6.1Parsingcommadelimitedlines
Note:Thisexampleisfordemonstrationpurposesonly.Ifyoureallytoneedtoparse
commadelimitedfields,youcanprobablydosomuchmoreeasilywiththeCSV
(commaseparatedvalues)moduleinthePythonstandardlibrary.
Hereisasimplegrammarforlinescontainingfieldsseparatedbycommas:
importsys
frompyparsingimportalphanums,ZeroOrMore,Word
Page100
fieldDef=Word(alphanums)
lineDef=fieldDef+ZeroOrMore(","+fieldDef)
deftest():
args=sys.argv[1:]
iflen(args)!=1:
print'usage:pythonpyparsing_test1.py<datafile.txt>'
sys.exit(1)
infilename=sys.argv[1]
forlineininfile:
fields=lineDef.parseString(line)
printfields
test()
Hereissomesampledata:
abcd,defg
11111,22222,33333
And,whenwerunourparseronthisdatafile,hereiswhatwesee:
$pythoncomma_parser.pysample1.data
['abcd',',','defg']
['11111',',','22222',',','33333']
NotehowthegrammarisconstructedfromnormalPythoncallstofunction
andobject/classconstructors.I'veconstructedtheparserinlinebecausemy
exampleissimple,butconstructingtheparserinafunctionorevenamodule
mightmakesenseformorecomplexgrammars.pyparsingmakesiteasytouse
thesethesedifferentstyles.
Use"+"tospecifyasequence.Inourexample,alineDefisafieldDef
followedby....
UseZeroOrMoretospecifyrepetition.Inourexample,alineDefisafieldDef
followedbyzeroormoreoccurancesofcommaandfieldDef.Thereisalso
OneOrMorewhenyouwanttorequireatleastoneoccurance.
Parsingcommadelimitedtexthappenssofrequentlythatpyparsingprovidesa
shortcut.Replace:
lineDef=fieldDef+ZeroOrMore(","+fieldDef)
with:
lineDef=delimitedList(fieldDef)
AndnotethatdelimitedListtakesanoptionalargumentdelimusedtospecify
thedelimiter.Thedefaultisacomma.
2.6.6.2Parsingfunctors
Thisexampleparsesexpressionsoftheformfunc(arg1, arg2, arg3):

frompyparsingimportWord,alphas,alphanums,nums,ZeroOrMore,
Page101
Literal
lparen=Literal("(")
rparen=Literal(")")
identifier=Word(alphas,alphanums+"_")
integer=Word(nums)
functor=identifier
arg=identifier|integer
args=arg+ZeroOrMore(","+arg)
expression=functor+lparen+args+rparen
deftest():
content=raw_input("Enteranexpression:")
parsedContent=expression.parseString(content)
printparsedContent
test()
Explanation:
UseLiteraltospecifyafixedstringthatistobematchedexactly.Inour
example,alparenisa(.
Wordtakesanoptionalsecondargument.Withasingle(string)argument,it
matchesanycontiguouswordmadeupofcharactersinthestring.Withtwo
(string)argumentsitmatchesawordwhosefirstcharacterisinthefirststring
andwhoseremainingcharactersareinthesecondstring.So,ourdefinitionof
identifiermatchesawordwhosefirstcharacterisanalphaandwhose
remainingcharactersarealphanumericsorunderscore.Asanotherexample,
youcanthinkofWord("0123456789")asanalogoustoaregexpcontainingthe
pattern"[09]+".
Useaverticalbarforalternation.Inourexample,anargcanbeeitheran
identifieroraninteger.
2.6.6.3Parsingnames,phonenumbers,etc.
Thisexampleparsesexpressionshavingthefollowingform:
Inputformat:
[name][phone][city,statezip]
Last,first1112223333city,ca99999
Hereistheparser:
importsys
frompyparsingimportalphas,nums,ZeroOrMore,Word,Group,
Suppress,Combine
lastname=Word(alphas)
firstname=Word(alphas)
city=Group(Word(alphas)+ZeroOrMore(Word(alphas)))
state=Word(alphas,exact=2)
zip=Word(nums,exact=5)
name=Group(lastname+Suppress(",")+firstname)
phone=Combine(Word(nums,exact=3)+""+Word(nums,exact=3)+
""+Word(nums,exact=4))
Page102
location=Group(city+Suppress(",")+state+zip)
record=name+phone+location
deftest():
args=sys.argv[1:]
iflen(args)!=1:
print'usage:pythonpyparsing_test3.py<datafile.txt>'
sys.exit(1)
infilename=sys.argv[1]
forlineininfile:
line=line.strip()
iflineandline[0]!="#":
fields=record.parseString(line)
printfields
test()
And,hereissomesampleinput:
Jabberer,Jerry1112223333Bakersfield,CA95111
Kackler,Kerry1112223334Fresno,CA95112
Louderdale,Larry1112223335LosAngeles,CA94001
Hereisoutputfromparsingtheaboveinput:
[['Jabberer','Jerry'],'1112223333',[['Bakersfield'],'CA',
'95111']]
[['Kackler','Kerry'],'1112223334',[['Fresno'],'CA',
'95112']]
[['Louderdale','Larry'],'1112223335',[['Los','Angeles'],
'CA','94001']]
Comments:
Weusethelen=nargumenttotheWordconstructortoresticttheparserto
acceptingaspecificnumberofcharacters,forexampleinthezipcodeand
phonenumber.Wordalsoacceptsmin=n'' and ``max=ntoenableyouto
restrictthelengthofawordtowithinarange.
WeuseGrouptogrouptheparsedresultsintosublists,forexampleinthe
definitionofcityandname.Groupenablesustoorganizetheparseresultsinto
simpleparsetrees.
WeuseCombinetojoinparsedresultsbackintoasinglestring.Forexample,
inthephonenumber,wecanrequiredashesandyetjointheresultsbackintoa
singlestring.
WeuseSuppresstoremoveunneededsubelementsfromparsedresults.For
example,wedonotneedthecommabetweenlastandfirstname.
2.6.6.4Amorecomplexexample
Thisexample(thankstoPaulMcGuire)parsesamorecomplexstructureandproduces
adictionary.
Hereisthecode:
Page103
frompyparsingimportLiteral,Word,Group,Dict,ZeroOrMore,
alphas,nums,\
delimitedList
importpprint
testData="""
++++++++++
||A1|B1|C1|D1|A2|B2|C2|D2|
+=======+======+======+======+======+======+======+======+======+
|min|7|43|7|15|82|98|1|37|
|max|11|52|10|17|85|112|4|39|
|ave|9|47|8|16|84|106|3|38|
|sdev|1|3|1|1|1|3|1|1|
++++++++++
"""
#Definegrammarfordatatable
heading=(Literal(
"+++++++++
+")+
"||A1|B1|C1|D1|A2|B2|C2|D2
|"+
"+=======+======+======+======+======+======+======+======+======+
").suppress()
vert=Literal("|").suppress()
number=Word(nums)
rowData=Group(vert+Word(alphas)+vert+
delimitedList(number,"|")+
vert)
trailing=Literal(
"+++++++++
+").suppress()
datatable=heading+Dict(ZeroOrMore(rowData))+trailing
defmain():
#Nowparsedataandprintresults
data=datatable.parseString(testData)
print"data:",data
print"data.asList():",
pprint.pprint(data.asList())
print"datakeys:",data.keys()
print"data['min']:",data['min']
print"data.max:",data.max
main()
Whenwerunthis,itproducesthefollowing:
data:[['min','7','43','7','15','82','98','1','37'],
['max','11','52','10','17','85','112','4','39'],
['ave','9','47','8','16','84','106','3','38'],
['sdev','1','3','1','1','1','3','1','1']]
data.asList():[['min','7','43','7','15','82','98','1',
'37'],
['max','11','52','10','17','85','112','4','39'],
['ave','9','47','8','16','84','106','3','38'],
['sdev','1','3','1','1','1','3','1','1']]
Page104
datakeys:['ave','min','sdev','max']
data['min']:['7','43','7','15','82','98','1','37']
data.max:['11','52','10','17','85','112','4','39']
Notes:
NotetheuseofDicttocreateadictionary.Theprintstatementsshowhowto
getattheitemsinthedictionary.
NotehowwecanalsogettheparseresultsasalistbyusingmethodasList.
Again,weusesuppresstoremoveunneededitemsfromtheparseresults.
2.7GUIApplications
2.7.1Introduction
ThissectionwillhelpyoutoputaGUI(graphicaluserinterface)inyourPython
program.
WewilluseaparticularGUIlibrary:PyGTK.We'vechosenthisbecauseitis
reasonablylightweightandourgoalistoembedlightweightGUIinterfacesinan
(possibly)existingapplication.
ForsimplerGUIneeds,considerEasyGUI,whichisalsodescribedbelow.
FormoreheavyweightGUIneeds(forexample,completeGUIapplications),you
maywanttoexploreWxPython.SeetheWxPythonhomepageat:
http://www.wxpython.org/
2.7.2PyGtk
InformationaboutPyGTKishere:ThePyGTKhomepagehttp://www.pygtk.org//.
2.7.2.1Asimplemessagedialogbox
InthissectionweexplainhowtopopupasimpledialogboxfromyourPython
application.
Todothis,dothefollowing:
1. ImportgtkintoyourPythonmodule.
2. Definethedialoganditsbehavior.
3. Createaninstanceofthedialog.
4. Runtheeventloop.
Hereisasamplethatdisplaysamessagebox:
importsys
importgetopt
importgtk
classMessageBox(gtk.Dialog):
def__init__(self,message="",buttons=(),pixmap=None,
Page105
modal=True):
gtk.Dialog.__init__(self)
self.connect("destroy",self.quit)
self.connect("delete_event",self.quit)
ifmodal:
self.set_modal(True)
hbox=gtk.HBox(spacing=5)
hbox.set_border_width(5)
self.vbox.pack_start(hbox)
hbox.show()
ifpixmap:
self.realize()
pixmap=Pixmap(self,pixmap)
hbox.pack_start(pixmap,expand=False)
pixmap.show()
label=gtk.Label(message)
hbox.pack_start(label)
label.show()
fortextinbuttons:
b=gtk.Button(text)
b.set_flags(gtk.CAN_DEFAULT)
b.set_data("user_data",text)
b.connect("clicked",self.click)
self.action_area.pack_start(b)
b.show()
self.ret=None
defquit(self,*args):
self.hide()
self.destroy()
gtk.main_quit()
defclick(self,button):
self.ret=button.get_data("user_data")
self.quit()
#createamessagebox,andreturnwhichbuttonwaspressed
defmessage_box(title="MessageBox",message="",buttons=(),
pixmap=None,
modal=True):
win=MessageBox(message,buttons,pixmap=pixmap,modal=modal)
win.set_title(title)
win.show()
gtk.main()
returnwin.ret
deftest():
result=message_box(title='Test#1',
message='Hereisyourmessage',
buttons=('Ok','Cancel'))
print'result:',result
USAGE_TEXT="""
Usage:
pythonsimple_dialog.py[options]
Options:
Example:
pythonsimple_dialog.py
"""
defusage():
Page106
printUSAGE_TEXT
sys.exit(1)
defmain():
args=sys.argv[1:]
try:
except:
usage()
relink=1
foropt,valinopts:
usage()
iflen(args)!=0:
usage()
test()
main()
Someexplanation:
First,weimportgtk
NextwedefineaclassMessageBoxthatimplementsamessagebox.Hereare
afewimportantthingstoknowaboutthatclass:
Itisasubclassofgtk.Dialog.
Itcreatesalabelandpacksitintothedialog'sclientarea.Notethata
DialogisaWindowthatcontainsavboxatthetopofandanaction_areaat
thebottomofitsclientarea.Theintensionisforustopackmiscellaneous
widgetsintothevboxandtoputbuttonssuchas"Ok","Cancel",etcinto
theaction_area.
Itcreatesonebuttonforeachbuttonlabelpassedtoitsconstructor.The
buttonsareallconnectedtotheclickmethod.
Theclickmethodsavesthevalueoftheuser_dataforthebuttonthatwas
clicked.Inourexample,thisvaluewillbeeither"Ok"or"Cancel".
And,wedefineafunction(message_box)that(1)createsaninstanceofthe
MessageBoxclass,(2)setsitstitle,(3)showsit,(4)startsitseventloopsothat
itcangetandprocesseventsfromtheuser,and(5)returnstheresulttothe
caller(inthiscase"Ok"or"Cancel").
Ourtestingfunction(test)callsfunctionmessage_boxandprintstheresult.
Thislookslikequiteabitofcode,untilyounoticethattheclassMessageBox
andthefunctionmessage_boxcouldbeputitautilitymoduleandreused.
2.7.2.2Asimpletextinputdialogbox
And,hereisanexamplethatdisplaysantextinputdialog:
importsys
importgetopt
importgtk
Page107
classEntryDialog(gtk.Dialog):
def__init__(self,message="",default_text='',modal=True):
gtk.Dialog.__init__(self)
ifmodal:
box=gtk.VBox(spacing=10)
box.set_border_width(10)
self.vbox.pack_start(box)
box.show()
ifmessage:
label=gtk.Label(message)
box.pack_start(label)
label.show()
self.entry=gtk.Entry()
self.entry.set_text(default_text)
box.pack_start(self.entry)
self.entry.show()
self.entry.grab_focus()
button=gtk.Button("OK")
button.connect("clicked",self.click)
button.set_flags(gtk.CAN_DEFAULT)
self.action_area.pack_start(button)
button.show()
button.grab_default()
button=gtk.Button("Cancel")
button.connect("clicked",self.quit)
button.set_flags(gtk.CAN_DEFAULT)
self.action_area.pack_start(button)
button.show()
self.ret=None
defquit(self,w=None,event=None):
self.hide()
self.destroy()
gtk.main_quit()
defclick(self,button):
self.ret=self.entry.get_text()
self.quit()
definput_box(title="InputBox",message="",default_text='',
modal=True):
win=EntryDialog(message,default_text,modal=modal)
win.show()
gtk.main()
returnwin.ret
deftest():
result=input_box(title='Test#2',
message='Enteravaluexxx:',
default_text='adefaultvalue')
ifresultisNone:
print'Canceled'
else:
print'result:"%s"'%result
USAGE_TEXT="""
Usage:
Page108
Options:
Example:
"""
defusage():
printUSAGE_TEXT
sys.exit(1)
defmain():
args=sys.argv[1:]
try:
except:
usage()
relink=1
foropt,valinopts:
usage()
iflen(args)!=0:
usage()
test()
main()
Mostoftheexplanationforthemessageboxexampleisrelevanttothisexample,too.
Herearesomedifferences:
OurEntryDialogclassconstructorcreatesinstanceofgtk.Entry,setsitsdefault
value,andpacksitintotheclientarea.
Theconstructoralsoautomaticallycreatestwobuttons:"OK"and"Cancel".
The"OK"buttonisconnecttotheclickmethod,whichsavesthevalueofthe
entryfield.The"Cancel"buttonisconnecttothequitmethod,whichdoesnot
savethevalue.
And,ifclassEntryDialogandfunctioninput_boxlookusableanduseful,add
themtoyourutilityguimodule.
2.7.2.3Afileselectiondialogbox
Thisexampleshowsafileselectiondialogbox:
importsys
importgetopt
importgtk
classFileChooser(gtk.FileSelection):
def__init__(self,modal=True,multiple=True):
gtk.FileSelection.__init__(self)
self.multiple=multiple
ifmodal:
Page109
self.cancel_button.connect('clicked',self.quit)
self.ok_button.connect('clicked',self.ok_cb)
ifmultiple:
self.set_select_multiple(True)
self.ret=None
defquit(self,*args):
self.hide()
self.destroy()
gtk.main_quit()
defok_cb(self,b):
ifself.multiple:
self.ret=self.get_selections()
else:
self.ret=self.get_filename()
self.quit()
deffile_sel_box(title="Browse",modal=False,multiple=True):
win=FileChooser(modal=modal,multiple=multiple)
win.show()
gtk.main()
returnwin.ret
deffile_open_box(modal=True):
returnfile_sel_box("Open",modal=modal,multiple=True)
deffile_save_box(modal=True):
returnfile_sel_box("SaveAs",modal=modal,multiple=False)
deftest():
result=file_open_box()
print'openresult:',result
result=file_save_box()
print'saveresult:',result
USAGE_TEXT="""
Usage:
Options:
Example:
"""
defusage():
printUSAGE_TEXT
sys.exit(1)
defmain():
args=sys.argv[1:]
try:
except:
usage()
relink=1
foropt,valinopts:
usage()
iflen(args)!=0:
usage()
Page110
test()
main()
#importpdb
#pdb.run('main()')
Alittleguidance:
Thereisapredefinedfileselectiondialog.Wesubclassit.
Thisexampledisplaysthefileselectiondialogtwice:oncewithatitle"Open"
andoncewithatitle"SaveAs".
Notehowwecancontrolwhetherthedialogallowsmultiplefileselections.
And,ifweselectthemultipleselectionmode,thenweuseget_selections
insteadofget_filenameinordertogettheselectedfilenames.
Thedialogcontainsbuttonsthatenabletheuserto(1)createanewfolder,(2)
deleteafile,and(3)renameafile.Ifyoudonotwanttheusertoperformthese
operations,thencallhide_fileop_buttons.Thiscalliscommentedoutinour
samplecode.
Notethattherearealsopredefineddialogsforfontselection(FontSelectionDialog)
andcolorselection(ColorSelectionDialog)
2.7.3EasyGUI
IfyourGUIneedsareminimalist(maybeapopupdialogortwo)andyour
applicationisimperativeratherthaneventdriven,thenyoumaywanttoconsider
EasyGUI.Asthenamesuggests,itisextremelyeasytouse.
HowtoknowwhenyoumightbeabletouseEasyGUI:
Yourapplicationdoesnotneedtoruninawindowcontainingmenusanda
menubar.
YourGUIneedsamounttolittlemorethandisplayingadialognowandthen
togetresponsesfromtheuser.
Youdonotwanttowriteaneventdrivenapplication,thatis,oneinwhich
yourcodesitsandwaitsforthetheusertoinitiateoperation,forexample,with
menuitems.
EasyGUIplusdocumentationandexamplesareavailableatEasyGUIhomepageat
SourceForgehttp://easygui.sourceforge.net/
EasyGUIprovidesfunctionsforavarietyofcommonlyneededdialogboxes,
including:
Amessageboxdisplaysamessage.
Ayes/nomessageboxdisplays"Yes"and"No"buttons.
Acontinue/cancelmessageboxdisplays"Continue"and"Cancel"buttons.
Achoiceboxdisplaysaselectionlist.
Anenterboxallowsentryofalineoftext.
Anintegerboxallowsentryofaninterger.
Amultipleentryboxallowsentryintomultiplefields.
Page111
Codeandtextboxessupportthedisplayoftextinmonospacedorporportional
fonts.
Fileanddirectoryboxesenabletheusertoselectafileoradirectory.
SeethedocumentationattheEasyGUIWebsiteformorefeatures.
ForademonstrationofEasyGUI'scapabilities,runtheeasygui.pyasaPythonscript:
$pythoneasygui.py
2.7.3.1AsimpleEasyGUIexample
Hereisasimpleexamplethatpromptstheuserforanentry,thenshowstheresponse
inamessagebox:
importeasygui
deftesteasygui():
response=easygui.enterbox(msg='Enteryourname:',
title='NameEntry')
easygui.msgbox(msg=response,title='YourResponse')
testeasygui()
2.7.3.2AnEasyGUIfileopendialogexample
Thisexamplepresentsadialogtoallowtheusertoselectafile:
importeasygui
deftest():
response=easygui.fileopenbox(msg='Selectafile')
print'filename:%s'%response
test()
2.8GuidanceonPackagesandModules
2.8.1Introduction
Pythonhasanexcellentrangeofimplementationorganizationstructures.Theserange
fromstatementsandcontrolstructures(atalowlevel)throughfunctions,methods,
andclasses(atanintermediatelevel)andmodulesandpackagesatanupperlevel.
Thissectionprovidessomeguidancewiththeuseofpackages.Inparticular:
Howtoconstructandimplementthem.
Howtousethem.
Howtodistributeandinstallthem.
2.8.2ImplementingPackages
APythonpackageisacollectionofPythonmodulesinadiskdirectory.
Page112
Inordertobeabletoimportindividualmodulesfromadirectory,thedirectorymust
containafilenamed__init__.py.(Notethatrequirementdoesnotapplytodirectories
thatarelistedinPYTHONPATH.)The__init__.pyservesseveralpurposes:
Thepresenceofthefile__init__.pyinadirectorymarksthedirectoryasa
Pythonpackage,whichenablesimportingmodulesfromthedirectory.
Thefirsttimeanapplicationimportsanymodulefromthedirectory/package,
thecodeinthemodule__init__isevaluated.
Ifthepackageitselfisimported(asopposedtoanindividualmodulewithin
thedirectory/package),thenitisthe__init__thatisimported(andevaluated).
2.8.3UsingPackages
Onesimplewaytoenabletheusertoimportanduseapackageistoinstructtheuseto
importindividualmodulesfromthepackage.
Asecond,slightlymoreadvancedwaytoenabletheusertoimportthepackageisto
exposethosefeaturesofthepackageinthe__init__module.Supposethatmodule
mod1containsfunctionsfun1aandfun1bandsupposethatmodulemod2contains
functionsfun2aandfun2b.Thenfile__init__.pymightcontainthefollowing:
frommod1importfun1a,fun1b
frommod2importfun2a,fun2b
Then,ifthefollowingisevaluatedintheuser'scode:
importtestpackages
Thentestpackageswillcontainfun1a,fun1b,fun2a,andfun2b.
Forexample,hereisaninteractivesessionthatdemostratesimportingthepackage:
>>>importtestpackages
>>>printdir(testpackages)
[`__builtins__',`__doc__',`__file__',`__name__',`__path__',
`fun1a',`fun1b',`fun2a',`fun2b',`mod1',`mod2']
2.8.4DistributingandInstallingPackages
Distutils(PythonDistributionUtilities)hasspecialsupportfordistrubutingand
installingpackages.Learnmorehere:DistributingPythonModules
http://docs.python.org/distutils/index.html.
Asourexample,imaginethatwehaveadirectorycontainingthefollowing:
Testpackages
Testpackages/README
Testpackages/MANIFEST.in
Testpackages/setup.py
Testpackages/testpackages/__init__.py
Testpackages/testpackages/mod1.py
Testpackages/testpackages/mod2.py
NoticethesubdirectoryTestpackages/testpackagescontainingthefile__init__.py.
Page113
ThisisthePythonpackagethatwewillinstall.
We'lldescribehowtoconfiguretheabovefilessothattheycanbepackagedasa
singledistributionfileandsothatthePythonpackagetheycontaincanbeinstalledas
apackagebyDistutils.
TheMANIFEST.infileliststhefilesthatwewantincludedinourdistribution.Here
isthecontentsofourMANIFEST.infile:
includeREADMEMANIFESTMANIFEST.in
includesetup.py
includetestpackages/*.py
Thesetup.pyfiledescribestoDistutils(1)howtopackagethedistributionfileand
(2)howtoinstallthedistribution.Hereisthecontentsofoursamplesetup.py:
fromdistutils.coreimportsetup#[1]
long_description='TestsforinstallinganddistributingPython
packages'
setup(name='testpackages',#[2]
version='1.0a',
description='TestsforPythonpackages',
maintainer='DaveKuhlman',
maintainer_email='dkuhlman@rexx.com',
url='http://www.rexx.com/~dkuhlman',
long_description=long_description,
packages=['testpackages']#[3]
)
Explanation:
1. WeimportthenecessarycomponentfromDistutils.
2. Wedescribethepackageanditsdeveloper/maintainer.
3. Wespecifythedirectorythatistobeinstalledasapackage.Whentheuser
installsourdistribution,thisdirectoryandallthemodulesinitwillbe
installedasapackage.
Now,tocreateadistributionfile,werunthefollowing:
pythonsetup.pysdistformats=gztar
whichwillcreateafiletestpackages-1.0a.tar.gzunderthedirectorydist.
Then,youcangivethisdistributionfiletoapotentialuser,whocaninstallitbydoing
thefollowing:
$tarxvzftestpackages1.0a.tar.gz
$cdtestpackages1.0a
$pythonsetup.pybuild
$pythonsetup.pyinstall#asroot
2.9EndMatter
Page114
2.9.1AcknowledgementsandThanks
ThankstotheimplementorsofPythonforproducinganexceptionallyusable
andenjoyableprogramminglanguage.
ThankstoDaveBeazleyandothersforSWIGandPLY.
ThankstoGregEwingforPyrexandPlex.
ThankstoJamesHenstridgeforPyGTK.
2.9.2SeeAlso
ThemainPythonWebSitehttp://www.python.orgformoreinformationon
Python.
PythonDocumentationhttp://www.python.org/doc/forlotsof
documentationonPython
Dave'sWebSitehttp://www.rexx.com/~dkuhlmanformoresoftwareand
informationonusingPythonforXMLandtheWeb.
TheSWIGhomepagehttp://www.swig.orgformoreinformationonSWIG
(SimplifiedWrapperandInterfaceGenerator).
ThePyrexhomepage
http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/formoreinformation
onPyrex.
PLY(PythonLexYacc)homepagehttp://www.dabeaz.com/ply/formore
informationonPLY.
ThePlexhomepage
http://www.cosc.canterbury.ac.nz/greg.ewing/python/Plex/formore
informationonPlex.
DistributingPythonModuleshttp://docs.python.org/distutils/index.htmlfor
informationonthePythonDistributionUtilities(Distutils).
Page115
3Part3PythonWorkbook
3.1Introduction
ThisdocumenttakesaworkbookandexercisewithsolutionsapproachtoPython
training.Itishopedthatthosewhofeelaneedforlessexplanationandmorepractical
exerciseswillfindthisuseful.
Afewnotesabouttheexercises:
I'vetriedtoincludesolutionsmostoftheexercises.Hopefully,youwillbe
abletocopyandpastethesesolutionsintoyourtexteditor,thenextendand
experimentwiththem.
IusetwointeractivePythoninterpreters(althoughtheyarethesamplePython
underneath).Whenyouseethisprompt>>>,it'sthestandardPython
interpreter.And,whenyouseethispromptIn [1]:,it'sIPython
http://ipython.scipy.org/moin/.
ThelatestversionofthisdocumentisatmyWebsite(URLabove).
Ifyouhavecommentsorsuggestions,pleasesendthemmyway.
3.2LexicalStructures
3.2.1Variablesandnames
Anameisanycombinationofletters,digits,andtheunderscore,butthefirstcharacter
mustbealetteroranunderscore.Namesmaybeofanylength.
Caseissignificant.
Exercises:
1. Whichofthefollowingarevalidnames?
1. total
2. total_of_all_vegetables
3. big-title-1
4. _inner_func
5. 1bigtitle
6. bigtitle1
Page116
2. Whichorthefollowingpairsarethesamename:
1. the_last_itemandthe_last_item
2. the_last_itemandThe_Last_Item
3. itemianditemj
4. item1anditeml
Solutions:
1. Items1,2,4,and6arevalid.Item3isnotasinglename,butisthreeitems
separatedbytheminusoperator.Item5isnotvalidbecauseitbeginswitha
digit.
2. Pythonnamesarecasesensitive,whichmeans:
1. the_last_itemandthe_last_itemarethesame.
2. the_last_itemandThe_Last_ItemaredifferentThesecondnamehas
anuppercasecharacters.
3. itemianditemjaredifferent.
4. item1anditemlaredifferentThisonemaybedifficulttosee,
dependingonthefontyouareviewing.Onenameendswiththedigitone;
theotherendswiththealphacharacter"el".Andthisexampleprovidesa
goodreasontouse"1"and"l"judiciouslyinnames.
ThefollowingarekeywordsinPythonandshouldnotbeusedasvariablenames:
anddelfromnotwhile
aselifglobalorwith
assertelseifpassyield
breakexceptimportprint
classexecinraise
continuefinallyisreturn
defforlambdatry
Exercises:
1. WhichofthefollowingarevalidnamesinPython?
1. _global
2. global
3. file
Solutions:
1. Donotusekeywordsforvariablenames:
1. Valid
2. Notavalidname."global"isakeyword.
3. Valid,however,"file"isthenameofabuiltintype,asyouwilllearnlater,
soyouareadvisednottoredefineit.Hereareafewofthenamesofbuilt
intypes:"file","int","str","float","list","dict",etc.SeeBuiltinTypes
http://docs.python.org/lib/types.htmlformorebuiltintypes..
ThefollowingareoperatorsinPythonandwillseparatenames:
+***///%
<<>>&|^~
<><=>===!=<>
andorisnotin
Page117
Also:()[].(dot)
But,notethatthePythonstyleguidesuggeststhatyouplaceblanksaroundbinary
operators.Oneexceptiontothisruleisfunctionargumentsandparametersfor
functions:itissuggestedthatyounotputblanksaroundtheequalsign(=)usedto
specifykeywordargumentsanddefaultparameters.
Exercises:
1. Whichofthefollowingaresinglenamesandwhicharenamesseparatedby
operators?
1. fruit_collection
2. fruit-collection
Solutions:
1. Donotuseadash,orotheroperator,inthemiddleofaname:
1. fruit_collectionisasinglename
2. fruit-collectionistwonamesseparatedbyadash.
3.2.2Linestructure
InPython,normallywewriteonestatementperline.Infact,Pythonassumesthis.
Therefore:
Statementseparatorsarenotnormallyneeded.
But,ifwewantmorethanonestatementonaline,weuseastatement
separator,specificallyasemicolon.
And,ifwewanttoextendastatementtoasecondorthirdlineandsoon,we
sometimesneedtodoabitextra.
ExtendingaPythonstatementtoasubsequentlineFollowthesetworules:
1. Ifthereisanopencontext,nothingspecialneedbedonetoextendastatement
acrossmultiplelines.Anopencontextisanopenparenthesis,anopensquare
bracket,oranopencurlybracket.
2. Wecanalwaysextendastatementonafollowinglinebyplacingabackslash
asthelastcharacteroftheline.
Exercises:
1. Extendthefollowingstatementtoasecondlineusingparentheses:
total_count=tree_count+vegetable_count+fruit_count
2. Extendthefollowingstatementtoasecondlineusingthebackslashline
continuationcharacter:
total_count=tree_count+vegetable_count+fruit_count
Solutions:
1. ParenthesescreateanopencontextthattellsPythonthatastatementextendsto
thenextline:
total_count=(tree_count+
Page118
vegetable_count+fruit_count)
2. AbackslashasthelastcharacteronlinetellsPythonthatthecurrentstatement
extendstothenextline:
total_count=tree_count+\
vegetable_count+fruit_count
Forextendingalineonasubsequentline,whichisbetter,parenthesesorabackslash?
Hereisaquote:
"ThepreferredwayofwrappinglonglinesisbyusingPython's
impliedlinecontinuationinsideparentheses,bracketsandbraces.If
necessary,youcanaddanextrapairofparenthesesaroundan
expression,butsometimesusingabackslashlooksbetter."
PEP8:StyleGuideforPythonCode
http://www.python.org/dev/peps/pep0008/
3.2.3Indentationandprogramstructure
Pythonusesindentationtoindicateprogramstructure.Thatistosay,inordertonesta
blockofcodeinsideacompoundstatement,youindentthatnestedcode.Thisis
differentfrommanyprogramminglanguageswhichusesomesortofbeginandend
markers,forexamplecurlybrackets.
ThestandardcodingpracticeforPythonistousefourspacesperindentationleveland
tonotusehardtabs.(SeetheStyleGuideforPythonCode.)Becauseofthis,youwill
wanttouseatexteditorthatyoucanconfiguresothatitwillusefourspacesfor
indentation.SeehereforalistofPythonfriendlytexteditors:PythonEditors.
Exercises:
1. Giventhefollowing,nesttheprintstatementinsidetheifstatement:
ifx>0:
printx
2. Nestthesetwolines:
z=x+y
printz
insidethefollowingfunctiondefinitionstatement:
defshow_sum(x,y):
Solutions:
1. Indentationindicatesthatonestatementisnestedinsideanotherstatement:
ifx>0:
printx
2. Indentationindicatesthatablockofstatementsisnestedinsideanother
statement:
Page119
defshow_sum(x,y):
z=x+y
printz
3.3ExecutionModel
Hereareafewrules:
1. PythonevaluatesPythoncodefromthetopofamoduledowntothebottomof
amodule.
2. Bindingstatementsattoplevelcreatenames(andbindvaluestothosenames)
asPythonevaluatescode.Furthermore,anameisnotcreateduntilitisbound
toavalue/object.
3. Anestedreferencetoaname(forexample,insideafunctiondefinitionorin
thenestedblockofanifstatement)isnotuseduntilthatnestedcodeis
evaluated.
Exercises:
1. Willthefollowingcodeproduceanerror?
show_version()
defshow_version():
print'Version1.0a'
2. Willthefollowingcodeproduceanerror?
deftest():
show_version()
defshow_version():
print'Version1.0a'
test()
3. Willthefollowingcodeproduceanerror?Assumethatshow_configisnot
defined:
x=3
ifx>5:
show_config()
Solutions:
1. Answer:Yes,itgeneratesanerror.Thenameshow_versionwouldnotbe
createdandboundtoavalueuntilthedeffunctiondefinitionstatementbindsa
functionobjecttoit.Thatisdoneaftertheattempttouse(call)thatobject.
2. Answer:No.Thefunctiontest()doescallthefunctionshow_version(),but
sincetest()isnotcalleduntilaftershow_version()isdefined,thatisOK.
3. Answer:No.It'sbadcode,butinthiscasewillnotgenerateanerror.Sincexis
lessthan5,thebodyoftheifstatementisnotevaluated.
N.B.Thisexampleshowswhyitisimportantduringtestingthateverylineof
codeinyourPythonprogrambeevaluated.HereisgoodPythonicadvice:"If
it'snottested,it'sbroken."
Page120
3.4BuiltinDataTypes
Eachofthesubsectionsinthissectiononbuiltindatatypeswillhaveasimilar
structure:
1. Abriefdescriptionofthedatatypeanditsuses.
2. RepresentationandconstructionHowtorepresentaninstanceofthedata
type.Howtocodealiteralrepresentationthatcreatesanddefinesaninstance.
Howtocreateaninstanceofthebuiltintype.
3. Operatorsthatareapplicabletothedatatype.
4. Methodsimplementedandsupportedbythedatatype.
3.4.1Numbers
Thenumbersyouwillusemostcommonlyarelikelytobeintegersandfloats.Python
alsohaslongintegersandcomplexnumbers.
Afewfactsaboutnumbers(inPython):
Pythonwillconverttousingalongintegerautomaticallywhenneeded.You
donotneedtoworryaboutexceedingthesizeofa(standard)integer.
ThesizeofthelargestintegerinyourversionofPythonisinsys.maxint.To
learnwhatitis,do:
>>>importsys
>>>printsys.maxint
9223372036854775807
Theaboveshowthemaximumsizeofanintegerona64bitversionof
Python.
Youcanconvertfromintegertofloatbyusingthefloatconstructor.Example:
>>>x=25
>>>y=float(x)
>>>printy
25.0
Pythondoes"mixedarithmetic".Youcanadd,multiply,anddivideintegers
andfloats.Whenyoudo,Python"promotes"theresulttoafloat.
3.4.1.1Literalrepresentationsofnumbers
Anintegerisconstructedwithaseriesofdigitsortheintegerconstructor(int(x)).Be
awarethatasequenceofdigitsbeginningwithzerorepresentsanoctalvalue.
Examples:
>>>x1=1234
>>>x2=int('1234')
>>>x3=25
>>>x1
1234
>>>x2
1234
>>>x3
25
Page121
Afloatisconstructedeitherwithdigitsandadot(example,12.345)orwith
engineering/scientificnotationorwiththefloatconstructor(float(x)).Examples:
>>>x1=2.0e3
>>>x1=1.234
>>>x2=1.234
>>>x3=float('1.234')
>>>x4=2.0e3
>>>x5=2.0e3
>>>printx1,x2,x3,x4,x5
1.2341.2341.2342000.00.002
Exercises:
Constructthesenumericvalues:
1.
2.
3.
4.
5.
6.
Integerzero
Floatingpointzero
Integeronehundredandone
Floatingpointonethousand
Floatingpointonethousandusingscientificnotation
Createapositiveinteger,anegativeinteger,andzero.Assignthemto
variables
7. Writeseveralarithmeticexpressions.Bindthevaluestovariables.Usea
varietyofoperators,e.g.+,-,/,*,etc.Useparenthesestocontroloperator
scope.
8. Createseveralfloatsandassignthemtovariables.
9. Writeseveralarithmeticexpressionscontainingyourfloatvariables.
10. Writeseveralexpressionsusingmixedarithmetic(integersandfloats).Obtain
afloatasaresultofdivisionofoneintegerbyanother;dosobyexplicitly
convertingoneintegertoafloat.
Solutions:
1.
2.
3.
4.
5.
6.
0
0.0,0.,or.0
101
1000.0
1e3or1.0e3
Asigningintegervaluestovariables:
In[7]:value1=23
In[8]:value2=14
In[9]:value3=0
In[10]:value1
Out[10]:23
In[11]:value2
Out[11]:14
In[12]:value3
Out[12]:0
7. Assigningexpressionvaluestovariables:
value1=4*(3+5)
value2=(value1/3.0)2
Page122
8. Assigningfloatstovariables:
value1=0.01
value2=3.0
value3=3e4
9. Assigningexpressionscontainingvarialbes:
value4=value1*(value2value3)
value4=value1+value2+value3value4
10. Mixedarithmetic:
x=5
y=8
z=float(x)/y
Youcanalsoconstructintegersandfloatsusingtheclass.Callingaclass(using
parenthesesafteraclassname,forexample)producesaninstanceoftheclass.
Exercises:
1. Constructanintegerfromthestring"123".
2. Constructafloatfromtheinteger123.
3. Constructanintegerfromthefloat12.345.
Solutions:
1. Usetheintdatatypetoconstructanintegerinstancefromastring:
int("123")
2. Usethefloatdatatypetoconstructafloatinstancefromaninteger:
float(123)
3. Usetheintdatatypetoconstructanintegerinstancefromafloat:
int(12.345)#>12
Noticethattheresultistruncatedtotheintegerpart.
3.4.1.2Operatorsfornumbers
Youcanusemostofthefamiliaroperatorswithnumbers,forexample:
+***///%
<<>>&|^~
<><=>===!=<>
Lookhereforanexplanationoftheseoperatorswhenappliedtonumbers:Numeric
Typesint,float,long,complexhttp://docs.python.org/lib/typesnumeric.html.
Someoperatorstakeprecedenceoverothers.ThetableintheWebpagejust
referencedabovealsoshowsthatorderofpriority.
Hereisabitofthattable:
Allnumerictypes(exceptcomplex)supportthefollowing
operations,
sortedbyascendingpriority(operationsinthesameboxhavethe
same
Page123
priority;allnumericoperationshaveahigherprioritythan
comparison
operations):
OperationResult
x+ysumofxandy
xydifferenceofxandy
x*yproductofxandy
x/yquotientofxandy
x//y(floored)quotientofxandy
x%yremainderofx/y
xxnegated
+xxunchanged
abs(x)absolutevalueormagnitudeofx
int(x)xconvertedtointeger
long(x)xconvertedtolonginteger
float(x)xconvertedtofloatingpoint
complex(re,im)acomplexnumberwithrealpartre,imaginarypart
im.imdefaultstozero.
c.conjugate()conjugateofthecomplexnumberc
divmod(x,y)thepair(x//y,x%y)
pow(x,y)xtothepowery
x**yxtothepowery
Noticealsothatthesameoperatormayperformadifferentfunctiondependingonthe
datatypeofthevaluetowhichitisapplied.
Exercises:
1. Addthenumbers3,4,and5.
2. Add2totheresultofmultiplying3by4.
3. Add2plus3andmultiplytheresultby4.
Solutions:
1. Arithmeticexpressionsarefollowstandardinfixalgebraicsyntax:
3+4+5
2. Useanotherinfixexpression:
2+3*4
Or:
2+(3*4)
But,inthiscasetheparenthesesarenotnecessarybecausethe*operatorbinds
moretightlythanthe+operator.
3. Useparenthesestocontrolorderofevaluation:
(2+3)*4
Notethatthe*operatorhasprecedenceover(bindstighterthan)the+
operator,sotheparenthesesareneeded.
Pythondoesmixedarithemetic.Whenyouapplyanoperationtoanintegeranda
float,itpromotestheresulttothe"higher"datatype,afloat.
Ifyouneedtoperformanoperationonseveralintegers,butwantuseafloatingpoint
Page124
operation,firstconvertoneoftheintegerstoafloatusingfloat(x),whicheffectively
createsaninstanceofclassfloat.
TrythefollowingatyourPythoninteractiveprompt:
1. 1.0 + 2
2. 2 / 3Noticethattheresultistruncated.
3. float(2) / 3Noticethattheresultisnottruncated.
Exercises:
1. Giventhefollowingassignments:
x=20
y=50
Dividexbyygivingafloatresult.
Solutions:
1. Promoteoneoftheintegerstofloatbeforeperformingthedivision:
z=float(x)/y
3.4.1.3Methodsonnumbers
Mostofthemethodsimplementedbythedatatypes(classes)intandfloatarespecial
methodsthatarecalledthroughtheuseofoperators.Specialmethodsoftenhave
namesthatbeginandendwithadoubleunderscore.Toseealistofthespecialnames
andabitofanindicationofwheneachiscalled,doanyofthefollowingatthePython
interactiveprompt:
>>>help(int)
>>>help(32)
>>>help(float)
>>>help(1.23)
>>>dir(1)
>>>dir(1.2)
3.4.2Lists
Listsareacontainerdatatypethatactsasadynamicarray.Thatistosay,alistisa
sequencethatcanbeindexedintoandthatcangrowandshrink.
Atupleisanindexablecontainer,likealist,exceptthatatupleisimmutable.
Afewcharacteristicsoflistsandtuples:
Alisthasa(current)lengthGetthelengthofalistwithlen(mylist).
AlisthasanorderTheitemsinalistareordered,andyoucanthinkofthat
orderasgoingfromlefttoright.
AlistisheterogeousYoucaninsertdifferenttypesofobjectsintothesame
list.
Listsaremutable,buttuplesarenot.Thus,thefollowingaretrueoflists,but
notoftuples:
Youcanextendedoraddtoalist.
Page125
Youcanshrinkalistbydeletingitemsfromit.
Youcaninsertitemsintothemiddleofalistoratthebeginningofalist.
Youcanadditemstotheendofalist.
Youcanchangewhichitemisatagivenpositioninalist.
3.4.2.1Literalrepresentationoflists
Theliteralrepresentationofalistissquarebracketscontainingzeroormoreitems
separatedbycommas.
Examples:
1. TrytheseatthePythoninteractiveprompt:
>>>[11,22,33]
>>>['aa','bb','cc',]
>>>[100,'apple',200,'banana',]#Thelastcommais
>>>optional.
2. Alistcancontainlists.Infactalistcancontainanykindofobject:
>>>[1,[2,3],4,[5,6,7,],8]
3. Listsareheterogenous,thatis,differentkindsofobjectscanbeinthesame
list.Hereisalistthatcontainsanumber,astring,andanotherlist:
>>>[123,'abc',[456,789]]
Exercises:
1. Create(define)thefollowingtuplesandlistsusingaliteral:
1. Atupleofintegers
2. Atupleofstrings
3. Alistofintegers
4. Alistofstrings
5. Alistoftuplesortupleoflists
6. Alistofintegersandstringsandtuples
7. Atuplecontainingexactlyoneitem
8. Anemptytuple
2. Doeachofthefollowing:
1. Printthelengthofalist.
2. PrinteachiteminthelistIterateovertheitemsinoneofyourlists.Print
eachitem.
3. Appendanitemtoalist.
4. Insertanitematthebeginningofalist.Insertaniteminthemiddleofa
list.
5. Addtwoliststogether.Dosobyusingboththeextendmethodandthe
plus(+)operator.Whatisthedifferencebetweenextendingalistand
addingtwolists?
6. Retrievethe2nditemfromoneofyourtuplesorlists.
7. Retrievethe2nd,3rd,and4thitems(aslice)fromoneofyourtuplesor
lists.
8. Retrievethelast(rightmost)iteminoneofyourlists.
Page126
9. Replaceaniteminalistwithanewitem.
10. Poponeitemofftheendofyourlist.
11. Deleteanitemfromalist.
12. Dothefollowinglistmanipulations:
1. Writeafunctionthattakestwoarguments,alistandanitem,andthat
appendstheitemtothelist.
2. Createanemptylist,
3. Callyourfunctionseveraltimestoappenditemstothelist.
4. Then,printouteachiteminthelist.
Solutions:
1. WecandefinelistliteralsatthePythonorIPythoninteractiveprompt:
1. Createatupleusingcommas,optionallywithparentheses:
In[1]:a1=(11,22,33,)
In[2]:a1
Out[2]:(11,22,33)
2. Quotedcharactersseparatedbycommascreateatupleofstrings:
In[3]:a2=('aaa','bbb','ccc')
In[4]:a2
Out[4]:('aaa','bbb','ccc')
3. Itemsseparatedbycommasinsidesquarebracketscreatealist:
In[26]:a3=[100,200,300,]
In[27]:a3
Out[27]:[100,200,300]
4. Stringsseparatedbycommasinsidesquarebracketscreatealistofstrings:
In[5]:a3=['basil','parsley','coriander']
In[6]:a3
Out[6]:['basil','parsley','coriander']
In[7]:
5. Atupleoralistcancontaintuplesandlists:
In[8]:a5=[(11,22),(33,44),(55,)]
In[9]:a5
Out[9]:[(11,22),(33,44),(55,)]
6. Alistortuplecancontainitemsofdifferenttypes:
In[10]:a6=[101,102,'abc',"def",(201,202),
('ghi','jkl')]
In[11]:a6
Out[11]:[101,102,'abc','def',(201,202),('ghi',
'jkl')]
7. Inordertocreateatuplecontainingexactlyoneitem,wemustusea
comma:
In[13]:a7=(6,)
In[14]:a7
Out[14]:(6,)
8. Inordertocreateanemptytuple,usethetupleclass/typetocreatean
instanceofaemptytuple:
Page127
In[21]:a=tuple()
In[22]:a
Out[22]:()
In[23]:type(a)
Out[23]:<type'tuple'>
3.4.2.2Operatorsonlists
Thereareseveraloperatorsthatareapplicabletolists.Hereishowtofindoutabout
them:
Dodir([])ordir(any_list_instance).Someoftheitemswithspecialnames
(leadingandtrainingdoubleunderscores)willgiveyoucluesaboutoperators
implementedbythelisttype.
Dohelp([])orhelp(list)atthePythoninteractiveprompt.
Dohelp(any_list_instance.some_method),wheresome_methodisoneof
theitemslistedusingdir(any_list_instance).
SeeSequenceTypesstr,unicode,list,tuple,buffer,xrange
http://docs.python.org/lib/typesseq.html
Exercises:
1. Concatenate(add)twoliststogether.
2. Createasinglelistthatcontainstheitemsinaninitiallistrepeated3times.
3. Comparetwolists.
Solutions:
1. Theplusoperator,appliedtotwolistsproducesanewlistthatisa
concatenationoftwolists:
>>>[11,22]+['aa','bb']
2. Multiplyingalistbyanintegerncreatesanewlistthatrepeatstheoriginallist
ntimes:
>>>[11,'abc',4.5]*3
3. Thecomparisonoperatorscanbeusedtocomparelists:
>>>[11,22]==[11,22]
>>>[11,22]<[11,33]
3.4.2.3Methodsonlists
Again,usedir()andhelp()tolearnaboutthemethodssupportedbylists.
Examples:
1. Createtwo(small)lists.Extendthefirstlistwiththeitemsinthesecond.
2. Appendseveralindividualitemstotheendofalist.
3. (a)Insertaitematthebeginningofalist.(b)Insertanitemsomewhereinthe
middleofalist.
4. Popanitemofftheendofalist.
Solutions:
Page128
1. Theextendmethodaddselementsfromanotherlist,orotheriterable:
>>>a=[11,22,33,44,]
>>>b=[55,66]
>>>a.extend(b)
>>>a
[11,22,33,44,55,66]
2. Usetheappendmethodonalisttoadd/appendanitemtotheendofalist:
>>>a=['aa',11]
>>>a.append('bb')
>>>a.append(22)
>>>a
['aa',11,'bb',22]
3. Theinsertmethodonalistenablesustoinsertitemsatagivenpositionina
list:
>>>a=[11,22,33,44,]
>>>a.insert(0,'aa')
>>>a
['aa',11,22,33,44]
>>>a.insert(2,'bb')
>>>a
['aa',11,'bb',22,33,44]
But,notethatweuseappendtoadditemsattheendofalist.
4. Thepopmethodonalistreturnsthe"rightmost"itemfromalistandremoves
thatitemfromthelist:
>>>a=[11,22,33,44,]
>>>
>>>b=a.pop()
>>>a
[11,22,33]
>>>b
44
>>>b=a.pop()
>>>a
[11,22]
>>>b
33
Notethattheappendandpopmethodstakentogethercanbeusedto
implementastack,thatisaLIFO(lastinfirstout)datastructure.
3.4.2.4Listcomprehensions
Alistcomprehensionisaconvenientwaytoproducealistfromaniterable(a
sequenceorotherobjectthatcanbeiteratedover).
Initssimplestform,alistcomprehensionresemblestheheaderlineofaforstatement
insidesquarebrackets.However,inalistcomprehension,theforstatementheaderis
prefixedwithanexpressionandsurroundedbysquarebrackets.Hereisatemplate:
[expr(x)forxiniterable]
where:
Page129
expr(x)isanexpression,usually,butnotalways,containingx.
iterableissomeiterable.Aniterablemaybeasequence(forexample,alist,a
string,atuple)oranunorderedcollectionoraniterator(somethingoverwhich
wecaniterateorapplyaforstatementto).
Hereisanexample:
>>>a=[11,22,33,44]
>>>b=[x*2forxina]
>>>b
[22,44,66,88]
Exercises:
1. Giventhefollowinglistofstrings:
names=['alice','bertrand','charlene']
producethefollowinglists:(1)alistofalluppercasenames;(2)alistof
capitalized(firstletteruppercase);
2. Giventhefollowingfunctionwhichcalculatesthefactorialofanumber:
deft(n):
ifn<=1:
returnn
else:
returnn*t(n1)
andthefollowinglistofnumbers:
numbers=[2,3,4,5]
createalistofthefactorialsofeachofthenumbersinthelist.
Solutions:
1. Forourexpressioninalistcomprehension,usetheupperandcapitalize
methods:
>>>names=['alice','bertrand','charlene']
>>>[name.upper()fornameinnames]
['ALICE','BERTRAND','CHARLENE']
>>>[name.capitalize()fornameinnames]
['Alice','Bertrand','Charlene']
2. Theexpressioninourlistcomprehensioncallsthefactorialfunction:
deft(n):
ifn<=1:
returnn
else:
returnn*t(n1)
deftest():
numbers=[2,3,4,5]
factorials=[t(n)forninnumbers]
print'factorials:',factorials
test()
Alistcomprehensioncanalsocontainanifclause.Hereisatemplate:
Page130
[expr(x)forxiniterableifpred(x)]
where:
pred(x)isanexpressionthatevaluatestoatrue/falsevalue.Valuesthatcount
asfalsearenumericzero,False,None,andanyemptycollection.Allother
valuescountastrue.
Onlyvaluesforwhichtheifclauseevaluatestotrueareincludedincreatingthe
resultinglist.
Examples:
>>>a=[11,22,33,44]
>>>b=[x*3forxinaifx%2==0]
>>>b
[66,132]
Exercises:
1. Giventwolists,generatealistofallthestringsinthefirstlistthatarenotin
thesecondlist.Herearetwosamplelists:
names1=['alice','bertrand','charlene','daniel']
names2=['bertrand','charlene']
Solutions:
1. Theifclauseofourlistcomprehensionchecksforcontainmentinthelist
names2:
deftest():
names1=['alice','bertrand','charlene','daniel']
names2=['bertrand','charlene']
names3=[namefornameinnames1ifnamenotinnames2]
print'names3:',names3
test()
Whenrun,thisscriptprintsoutthefollowing:
names3:['alice','daniel']
3.4.3Strings
Astringisanorderedsequenceofcharacters.Hereareafewcharacteristicsofstrings:
Astringhasalength.Getthelengthwiththelen()builtinfunction.
Astringisindexable.Getasinglecharacteratapositioninastringwiththe
squarebracketoperator,forexamplemystring[5].
Youcanretrieveaslice(substring)ofastringwithasliceoperation,for
examplemystring[5:8].
Createstringswithsinglequotesordoublequotes.Youcanputsinglequotesinside
doublequotesandyoucanputdoublequotesinsidesinglequotes.Youcanalso
escapecharacterswithabackslash.
Exercises:
Page131
1. Createastringcontainingasinglequote.
2. Createastringcontainingadoublequote.
3. Createastringcontainingbothasinglequoteadoublequote.
Solutions:
1. Createastringwithdoublequotestoincludesinglequotesinsidethestring:
>>>str1="thatisjerry'sball"
2. Createastringwithsinglequotestoincludesinglequotesinsidethestring:
>>>str1='say"goodbye",bullwinkle'
3. Takeyourchoice.Escapeeitherthesinglequotesorthedoublequoteswitha
backslash:
>>>str1='say"hello"tojerry\'smom'
>>>str2="say\"hello\"tojerry'smom"
>>>str1
'say"hello"tojerry\'smom'
>>>str2
'say"hello"tojerry\'smom'
Triplequotesenableyoutocreateastringthatspansmultiplelines.Usethreesingle
quotesorthreedoublequotestocreateasinglequotedstring.
Examples:
1. Createatriplequotedstringthatcontainssingleanddoublequotes.
Solutions:
1. Usetriplesinglequotesortripledoublequotestocreatemultilinestrings:
String1='''Thisstringextends
acrossseverallines.And,soithas
endoflinecharactersinit.
'''
String2="""
Thisstringbeginsandendswithanendofline
character.Itcanhaveboth'single'
quotesand"double"quotesinit.
"""
deftest():
printString1
printString2
test()
3.4.3.1Characters
Pythondoesnothaveadistinctcharactertype.InPython,acharacterisastringof
length1.Youcanusetheord()andchr()builtinfunctionstoconvertfromcharacter
tointegerandback.
Exercises:
Page132
1. Createacharacter"a".
2. Createacharacter,thenobtainitsintegerrepresentation.
Solutions:
1. Thecharacter"a"isaplainstringoflength1:
>>>x='a'
2. Theintegerequivalentoftheletter"A":
>>>x="A"
>>>ord(x)
65
3.4.3.2Operatorsonstrings
Youcanconcatenatestringswiththe"+"operator.
Youcancreatemultipleconcatenatedcopiesofastringwiththe"*"operator.
And,augmentedassignment(+=and*=)alsowork.
Examples:
>>>'cat'+'and'+'dog'
'catanddog'
>>>'#'*40
'########################################'
>>>
>>>s1='flower'
>>>s1+='s'
>>>s1
'flowers'
Exercises:
1. Giventhesestrings:
>>>s1='abcd'
>>>s2='efgh'
createanewstringcomposedofthefirststringfollowedby(concatenated
with)thesecond.
2. Createasinglestringcontaining5copiesofthestring'abc'.
3. Usethemultiplicationoperatortocreatea"line"of50dashes.
4. Herearethecomponentsofapathtoafileonthefilesystem:"home",
"myusername","Workdir","notes.txt".Concatenatethesetogetherseparating
themwiththepathseparatortoformacompletepathtothatfile.(Notethatif
youusethebackslashtoseparatecomponentsofthepath,youwillneedtouse
adoublebackslash,becausethebackslashistheescapecharacterinstrings.
Solutions:
1. Theplus(+)operatorappliedtoastringcanbeusedtoconcatenatestrings:
>>>s3=s1+s2
>>>s3
'abcdefgh'
Page133
2. Themultiplicationoperator(*)appliedtoastringcreatesanewstringthat
concatenatesastringwithitselfsomenumberoftimes:
>>>s1='abc'*5
>>>s1
'abcabcabcabcabc'
3. Themultiplicationoperator(*)appliedtoastringcanbeusedtocreatea
"horizontaldividerline":
>>>s1=''*50
>>>prints1
4. Thesepmemberoftheosmodulegivesusaplatformindependentwayto
constructpaths:
>>>importos
>>>
>>>a=["home","myusername","Workdir","notes.txt"]
>>>path=a[0]+os.sep+a[1]+os.sep+a[2]+os.sep+
a[3]
>>>path
'home/myusername/Workdir/notes.txt'
And,amoreconcisesolution:
>>>importos
>>>a=["home","myusername","Workdir","notes.txt"]
>>>os.sep.join(a)
'home/myusername/Workdir/notes.txt'
Notes:
Notethatimportingtheosmoduleandthenusingos.sepfromthat
modulegivesusaplatformindependentsolution.
Ifyoudodecidetocodethepathseparatorcharacterexplicitlyandifyou
areonMSWindowswherethepathseparatoristhebackslash,thenyou
willneedtouseadoublebackslash,becausethatcharacteristheescape
character.
3.4.3.3Methodsonstrings
Stringsupportavarietyofoperations.Youcanobtainalistofthesemethodsbyusing
thedir()builtinfunctiononanystring:
>>>dir("")
['__add__','__class__','__contains__','__delattr__','__doc__',
'__eq__','__ge__','__getattribute__','__getitem__',
'__getnewargs__','__getslice__','__gt__','__hash__',
'__init__',
'__le__','__len__','__lt__','__mod__','__mul__','__ne__',
'__new__','__reduce__','__reduce_ex__','__repr__','__rmod__',
'__rmul__','__setattr__','__str__','capitalize','center',
'count','decode','encode','endswith','expandtabs','find',
'index','isalnum','isalpha','isdigit','islower','isspace',
'istitle','isupper','join','ljust','lower','lstrip',
'partition','replace','rfind','rindex','rjust','rpartition',
'rsplit','rstrip','split','splitlines','startswith','strip',
Page134
'swapcase','title','translate','upper','zfill']
And,youcangethelponanyspecificmethodbyusingthehelp()builtinfunction.
Hereisanexample:
>>>help("".strip)
Helponbuiltinfunctionstrip:
strip(...)
S.strip([chars])>stringorunicode
ReturnacopyofthestringSwithleadingandtrailing
whitespaceremoved.
IfcharsisgivenandnotNone,removecharactersinchars
instead.
Ifcharsisunicode,Swillbeconvertedtounicodebefore
stripping
Exercises:
1. Stripallthewhitespacecharactersofftherightendofastring.
2. Centerashortstringwithinalongerstring,thatis,padashortstringwith
blankcharactersonbothrightandlefttocenterit.
3. Convertastringtoalluppercase.
4. Splitastringintoalistof"words".
5. (a)Jointhestringsinalistofstringstoformasinglestring.(b)Ditto,butput
anewlinecharacterbetweeneachoriginalstring.
Solutions:
1. Therstrip()methodstripswhitespaceofftherightsideofastring:
>>>s1='sometext\n'
>>>s1
'sometext\n'
>>>s2=s1.rstrip()
>>>s2
'sometext'
2. Thecenter(n)methodcentersastringwithinapaddedstringofwidthn:
>>>s1='Dave'
>>>s2=s1.center(20)
>>>s2
'Dave'
3. Theupper()methodproducesanewstringthatconvertsallalphacharacters
intheoriginaltouppercase:
>>>s1='Banana'
>>>s1
'Banana'
>>>s2=s1.upper()
>>>s2
'BANANA'
4. Thesplit(sep)methodproducesalistofstringsthatareseparatedbysepin
theoriginalstring.Ifsepisomitted,whitespaceistreatedastheseparator:
>>>s1="""howdoesitfeel
Page135
...tobeonyourown
...nodirectionsknown
...likearollingstone
..."""
>>>words=s1.split()
>>>words
['how','does','it','feel','to','be','on','your',
'own','no',
'directions','known','like','a','rolling','stone']
Notethatthesplit()functioninthere(regularexpression)moduleisuseful
whentheseparatorismorecomplexthanwhitespaceorasinglecharacter.
5. Thejoin()methodconcatenatesstringsfromalistofstringstoformasingle
string:
>>>lines=[]
>>>lines.append('howdoesitfeel')
>>>lines.append('tobeonyourown')
>>>lines.append('nodirectionsknown')
>>>lines.append('likearollingstone')
>>>lines
['howdoesitfeel','tobeonyourown','nodirections
known',
'likearollingstone']
>>>s1=''.join(lines)
>>>s2=''.join(lines)
>>>s3='\n'.join(lines)
>>>s1
'howdoesitfeeltobeonyourownnodirectionsknownlikea
rollingstone'
>>>s2
'howdoesitfeeltobeonyourownnodirectionsknownlike
arollingstone'
>>>s3
'howdoesitfeel\ntobeonyourown\nnodirections
known\nlikearollingstone'
>>>prints3
howdoesitfeel
tobeonyourown
nodirectionsknown
likearollingstone
3.4.3.4Rawstrings
Rawstringsgiveusaconvenientwaytoincludethebackslashcharacterinastring
withoutescaping(withanadditionalbackslash).Rawstringslooklikeplainliteral
strings,butareprefixedwithan"r"or"R".SeeStringliterals
http://docs.python.org/reference/lexical_analysis.html#stringliterals
Excercises:
1. Createastringthatcontainsabackslashcharacterusingbothplainliteral
stringandarawstring.
Solutions:
1. Weusean"r"prefixtodefinearawstring:
>>>print'abc\\def'
Page136
abc\def
>>>printr'abc\def'
abc\def
3.4.3.5Unicodestrings
Unicodestringsgiveusaconsistentwaytoprocesscharacterdatafromavarietyof
characterencodings.
Excercises:
1. Createseveralunicodestrings.Useboththeunicodeprefixcharacter("u")and
theunicodetype(unicode(some_string)).
2. Convertastring(possiblyfromanothernonasciiencoding)tounicode.
3. Convertaunicodestringtoanotherencoding,forexample,utf8.
4. Testastringtodetermineifitisunicode.
5. Createastringthatcontainsaunicodecharacter,thatis,acharacteroutsidethe
asciicharacterset.
Solutions:
1. Wecanrepresentunicodestringwitheitherthe"u"prefixorwithacalltothe
unicodetype:
defexercise1():
a=u'abcd'
printa
b=unicode('efgh')
printb
2. Weconvertastringfromanothercharacterencodingintounicodewiththe
decode()stringmethod:
importsys
defexercise2():
a='abcd'.decode('utf8')
printa
b='abcd'.decode(sys.getdefaultencoding())
printb
3. Wecanconvertaunicodestringtoanothercharacterencodingwiththe
encode()stringmethod:
importsys
defexercise3():
a=u'abcd'
printa.encode('utf8')
printa.encode(sys.getdefaultencoding())
4. Herearetwowaystocheckthetypeofastring:
importtypes
defexercise4():
a=u'abcd'
printtype(a)istypes.UnicodeType
printtype(a)istype(u'')
Page137
5. Wecanencodeunicodecharactersinastringinseveralways,forexample,(1)
bydefiningautf8stringandconvertingittounicodeor(2)definingastring
withanembeddedunicodecharacteror(3)concatenatingaunicodecharacher
intoastring:
defexercise5():
utf8_string='IvanKrsti\xc4\x87'
unicode_string=utf8_string.decode('utf8')
printunicode_string.encode('utf8')
printlen(utf8_string)
printlen(unicode_string)
unicode_string=u'aa\u0107bb'
unicode_string='aa'+unichr(263)+'bb'
Guidanceforuseofencodingsandunicode:
1. Convert/decodefromanexternalencodingtounicodeearly:
my_source_string.decode(encoding)
2. Doyourwork(Pythonprocessing)inunicode.
3. Convert/encodetoanexternalencodinglate(forexample,justbeforesaving
toanexternalfile):
my_unicode_string.encode(encoding)
Formoreinformation,see:
UnicodeInPython,CompletelyDemystified
http://farmdev.com/talks/unicode/
PEP100:PythonUnicodeIntegration
http://www.python.org/dev/peps/pep0100/
4.8codecsCodecregistryandbaseclasses
http://docs.python.org/lib/modulecodecs.html
4.8.2EncodingsandUnicodehttp://docs.python.org/lib/encodings
overview.html
4.8.3StandardEncodingshttp://docs.python.org/lib/standard
encodings.html
3.4.4Dictionaries
Adictionaryisanunorderedcollectionofkeyvaluepairs.
Adictionaryhasalength,specificallythenumberofkeyvaluepairs.
Thekeysmustbeimmutableobjecttypes.
3.4.4.1Literalrepresentationofdictionaries
Curleybracketsareusedtorepresentadictionary.Eachpairinthedictionaryis
representedbyakeyandvalueseparatedbyacolon.Multiplepairsareseparatedby
comas.Forexample,hereisanemptydictionaryandseveraldictionariescontaining
key/valuepairs:
Page138
In[4]:d1={}
In[5]:d2={'width':8.5,'height':11}
In[6]:d3={1:'RED',2:'GREEN',3:'BLUE',}
In[7]:d1
Out[7]:{}
In[8]:d2
Out[8]:{'height':11,'width':8.5}
In[9]:d3
Out[9]:{1:'RED',2:'GREEN',3:'BLUE'}
Notes:
Acommaafterthelastpairisoptional.SeetheREDGREENBLUEexample
above.
Stringsandintegersworkaskeys,sincetheyareimmutable.Youmightalso
wanttothinkabouttheuseoftuplesofintegersaskeysinadictionaryusedto
representasparsearray.
Exercises:
1. Defineadictionarythathasthefollowingkeyvaluepairs:
2. Defineadictionarytorepresentthe"enum"daysoftheweek:Sunday,
Monday,Tuesday,...
Solutions:
1. Adictionarywhosekeysandvaluesarestringscanbeusedtorepresentthis
table:
vegetables={
'Eggplant':'Purple',
'Tomato':'Red',
'Parsley':'Green',
'Lemon':'Yellow',
'Pepper':'Green',
}
Notethattheopencurlybracketenablesustocontinuethisstatementacross
multiplelineswithoutusingabackslash.
2. Wemightusestringsforthenamesofthedaysoftheweekaskeys:
DAYS={
'Sunday':1,
'Monday':2,
'Tuesday':3,
'Wednesday':4,
'Thrusday':5,
'Friday':6,
'Saturday':7,
}
3.4.4.2Operatorsondictionaries
Dictionariessupportthefollowing"operators":
Lengthlen(d)returnsthenumberofpairsinadictionary.
IndexingYoucanbothsetandgetthevalueassociatedwithakeybyusing
theindexingoperator[ ].Examples:
Page139
In[12]:d3[2]
Out[12]:'GREEN'
In[13]:d3[0]='WHITE'
In[14]:d3[0]
Out[14]:'WHITE'
TestforkeyTheinoperatortestsfortheexistenceofakeyinadictionary.
Example:
In[6]:trees={'poplar':'deciduous','cedar':
'evergreen'}
In[7]:if'cedar'intrees:
...:print'Thecedaris%s'%(trees['cedar'],)
...:
Thecedarisevergreen
Exercises:
1. Createanemptydictionary,thenusetheindexingoperator[ ]toinsertthe
followingnamevaluepairs:
"red""255:0:0"
"green""0:255:0"
"blue""0:0:255"
2. Printoutthenumberofitemsinyourdictionary.
Solutions:
1. Wecanuse"[]"tosetthevalueofakeyinadictionary:
deftest():
colors={}
colors["red"]="255:0:0"
colors["green"]="0:255:0"
colors["blue"]="0:0:255"
print'Thevalueofredis"%s"'%(colors['red'],)
print'Thecolorsdictionarycontains%ditems.'%
(len(colors),)
test()
Whenwerunthis,wesee:
Thevalueofredis"255:0:0"
Thecolorsdictionarycontains3items.
2. Thelen()builtinfunctiongivesusthenumberofitemsinadictionary.See
theprevioussolutionforanexampleofthis.
3.4.4.3Methodsondictionaries
Hereisatablethatdescribesthemethodsapplicabletodictionarys:
Operation
Result
len(a)
thenumberofitemsina
a[k]
theitemofawithkeyk
a[k]=v
seta[k]tov
Page140
Operation
Result
dela[k]
removea[k]froma
a.clear()
removeallitemsfroma
a.copy()
a(shallow)copyofa
kina
Trueifahasakeyk,elseFalse
knotina
equivalenttonotkina
a.has_key(k)
equivalenttokina,usethatforminnewcode
a.items()
acopyofa'slistof(key,value)pair
a.keys()
acopyofa'slistofkeys
a.update([b])
updatesawithkey/valuepairsfromb,overwriting
existingkeys,returnsNone
a.fromkeys(seq[,value])
createsanewdictionarywithkeysfromseqandvalues
settovalue
a.values()
acopyofa'slistofvalues
a.get(k[,x])
a[k]ifkina,elsex)
a.setdefault(k[,x])
a[k]ifkina,elsex(alsosettingit)
a.pop(k[,x])
a[k]ifkina,elsex(andremovek)(8)
a.popitem()
removeandreturnanarbitrary(key,value)pair
a.iteritems()
returnaniteratorover(key,value)pairs
a.iterkeys()
returnaniteratoroverthemapping'skeys
a.itervalues()
returnaniteratoroverthemapping'svalues
YoucanalsofindthistableatthestandarddocumentationWebsiteinthe"Python
LibraryReference":MappingTypesdict
http://docs.python.org/lib/typesmapping.html
Exercises:
1. Printthekeysandvaluesintheabove"vegetable"dictionary.
2. Printthekeysandvaluesintheabove"vegetable"dictionarywiththekeysin
alphabeticalorder.
3. Testfortheoccuranceofakeyinadictionary.
Solutions:
1. Wecanusethed.items()methodtoretrievealistoftuplescontainingkey
Page141
valuepairs,thenuseunpackingtocapturethekeyandvalue:
Vegetables={
'Tomato':'Red',
'Parsley':'Green',
'Lemon':'Yellow',
'Pepper':'Green',
}
deftest():
forkey,valueinVegetables.items():
print'key:',key,'value:',value
test()
2. Weretrievealistofkeyswiththekeys()method,thesortitwiththelistsort()
method:
Vegetables={
'Tomato':'Red',
'Parsley':'Green',
'Lemon':'Yellow',
'Pepper':'Green',
}
deftest():
keys=Vegetables.keys()
keys.sort()
forkeyinkeys:
print'key:',key,'value:',Vegetables[key]
test()
3. Totestfortheexistenceofakeyinadictionary,wecanuseeitherthein
operator(preferred)orthed.has_key()method(oldstyle):
Vegetables={
'Tomato':'Red',
'Parsley':'Green',
'Lemon':'Yellow',
'Pepper':'Green',
}
deftest():
if'Eggplant'inVegetables:
print'wehave%segplants'%Vegetables['Eggplant']
if'Banana'notinVegetables:
print'yeswehavenobananas'
ifVegetables.has_key('Parsley'):
print'wehaveleafy,%sparsley'%
Vegetables['Parsley']
test()
Whichwillprintout:
wehavePurpleegplants
yeswehavenobananas
Page142
wehaveleafy,Greenparsley
3.4.5Files
APythonfileobjectrepresentsafileonafilesystem.
Afileobjectopenforreadingatextfileisiterable.Whenweiterateoverit,it
producesthelinesinthefile.
Afilemaybeopenedinthesemodes:
'r'readmode.Thefilemustexist.
'w'writemode.Thefileiscreated;anexistingfileisoverwritten.
'a'appendmode.Anexistingfileisopenedforwriting(attheendofthe
file).Afileiscreatedifitdoesnotexist.
Theopen()builtinfunctionisusedtocreateafileobject.Forexample,thefollowing
code(1)opensafileforwriting,then(2)forreading,then(3)forappending,and
finally(4)forreadingagain:
deftest(infilename):
#1.Openthefileinwritemode,whichcreatesthefile.
outfile=open(infilename,'w')
outfile.write('line1\n')
outfile.close()
#2.Openthefileforreading.
forlineininfile:
print'Line:',line.rstrip()
infile.close()
#3.Openthefileinappendmode,andaddalinetotheend
of
#thefile.
outfile=open(infilename,'a')
outfile.close()
print''*40
#4.Openthefileinreadmodeoncemore.
forlineininfile:
print'Line:',line.rstrip()
infile.close()
test('tmp.txt')
Exercises:
1. Openatextfileforreading,thenreadtheentirefileasasinglestring,andthen
splitthecontentonnewlinecharacters.
2. Openatextfileforreading,thenreadtheentirefileasalistofstrings,where
eachstringisonelineinthefile.
3. Openatextfileforreading,theniterateofeachlineinthefileandprintitout.
Solutions:
1. Usetheopen()builtinfunctiontoopenthefileandcreateafileobject.Use
Page143
theread()methodonthefileobjecttoreadtheentirefile.Usethesplit()or
splitlines()methodstosplitthefileintolines:
>>>infile=open('tmp.txt','r')
>>>content=infile.read()
>>>infile.close()
>>>lines=content.splitlines()
>>>printlines
['line1','line2','line3','']
2. Thef.readlines()methodreturnsalistoflinesinafile:
>>>infile=open('tmp.txt','r')
>>>lines=infile.readlines()
>>>infile.close()
>>>printlines
['line1\n','line2\n','line3\n']
3. Sinceafileobject(openforreading)isitselfaniterator,wecaniterateoverit
inaforstatement:
"""
Testiterationoveratextfile.
Usage:
pythontest.pyin_file_name
"""
importsys
forlineininfile:
#Stripoffthenewlinecharacterandany
whitespaceon
#theright.
line=line.rstrip()
#Printonlynonblanklines.
ifline:
printline
infile.close()
defmain():
args=sys.argv[1:]
iflen(args)!=1:
print__doc__
sys.exit(1)
infilename=args[0]
test(infilename)
main()
Notes:
Thelasttwolinesofthissolutioncheckthe__name__attributeofthe
moduleitselfsothatthemodulewillrunasascriptbutwillnotrunwhen
themoduleisimportedbyanothermodule.
The__doc__attributeofthemodulegivesusthemodule'sdocstring,
whichisthestringdefinedatthetopofthemodule.
sys.argvgivesusthecommandline.And,sys.argv[1:]chopsoffthe
Page144
programname,leavinguswiththecommanlinearguments.
3.4.6Afewmiscellaneousdatatypes
3.4.6.1None
Noneisasingleton.ThereisonlyoneinstanceofNone.Usethisvaluetoindicate
theabsenceofanyother"real"value.
TestforNonewiththeidentityoperatoris.
Exercises:
1. Createalist,someofwhoseelementsareNone.Thenwriteaforloopthat
countsthenumberofoccurancesofNoneinthelist.
Solutions:
1. Theidentityoperatorsisandis notcanbeusedtotestforNone:
>>>a=[11,None,'abc',None,{}]
>>>a
[11,None,'abc',None,{}]
>>>count=0
>>>foritemina:
...ifitemisNone:
...count+=1
...
>>>
>>>printcount
2
3.4.6.2ThebooleansTrueandFalse
PythonhasthetwobooleanvaluesTrueandFalse.Manycomparisonoperators
returnTrueandFalse.
Examples:
1. Whatvalueisreturnedby3 > 2?
Answer:ThebooleanvalueTrue.
2. Giventhesevariabledefinitions:
x=3
y=4
z=5
Whatdoesthefollowingprintout:
printy>xandz>y
AnswerPrintsout"True"
3.5Statements
Page145
3.5.1Assignmentstatement
Theassignmentstatementusestheassignmentoperator=.
Theassignmentstatementisabindingstatement:itbindsavaluetoanamewithina
namespace.
Exercises:
1. Bindthevalue"eggplant"tothevariablevegetable.
Solutions:
1.The=operatorisanassignmentstatementthatbindsavaluetoavariable:
>>>vegetable="eggplant"
Thereisalsoaugmentedassignmentusingtheoperators+=,-=,*=,/=,etc.
Exercises:
1. Useaugmentedassignmenttoincrementthevalueofaninteger.
2. Useaugmentedassignmenttoappendcharacterstotheendofastring.
3. Useaugmentedassignmenttoappendtheitemsinonelisttoanother.
4. Useaugmentedassignmenttodecrementavariablecontaininganintegerby1.
Solutions:
1. The+=operatorincrementsthevalueofaninteger:
>>>count=0
>>>count+=1
>>>count
1
>>>count+=1
>>>count
2
2. The+=operatorappendscharacterstotheendofastring:
>>>buffer='abcde'
>>>buffer+='fgh'
>>>buffer
'abcdefgh'
3. The+=operatorappendsitemsinonelisttoanother:
In[20]:a=[11,22,33]
In[21]:b=[44,55]
In[22]:a+=b
In[23]:a
Out[23]:[11,22,33,44,55]
1. The-=operatordecrementsthevalueofaninteger:
>>>count=5
>>>count
5
>>>count=1
>>>count
4
Youcanalsoassignavalueto(1)anelementofalist,(2)aniteminadictionary,(3)
Page146
anattributeofanobject,etc.
Exercises:
1. Createalistofthreeitems,thenassignanewvaluetothe2ndelementinthe
list.
2. Createadictionary,thenassignvaluestothekeys"vegetable"and"fruit"in
thatdictionary.
3. Usethefollowingcodetocreateaninstanceofaclass:
classA(object):
pass
a=A()
Thenassignvaluestoanattribuenamedcategoryinthatinstance.
Solutions:
1. Assignmentwiththeindexingoperator[]assignsavaluetoanelementina
list:
>>>trees=['pine','oak','elm']
>>>trees
['pine','oak','elm']
>>>trees[1]='cedar'
>>>trees
['pine','cedar','elm']
2. Assignmentwiththeindexingoperator[]assignsavaluetoanitem(akey
valuepair)inadictionary:
>>>foods={}
>>>foods
{}
>>>foods['vegetable']='greenbeans'
>>>foods['fruit']='nectarine'
>>>foods
{'vegetable':'greenbeans','fruit':'nectarine'}
3. Assignmentalongwiththedereferencingoperator.(dot)enablesustoassign
avaluetoanattributeofanobject:
>>>classA(object):
...pass
...
>>>a=A()
>>>a.category=25
>>>a.__dict__
{'category':25}
>>>a.category
25
3.5.2printstatement
Warning:BeawarethattheprintstatementwillgoawayinPythonversion3.0.It
willbereplacedbythebuiltinprint()function.
Theprintstatementsendsoutputtostandardoutput.Itprovidesasomewhatmore
convenientwayofproducingoutputthanusingsys.stdout.write().
Page147
Theprintstatementtakesaseriesofzeroormoreobjectsseparatedbycommas.Zero
objectsproducesablankline.
Theprintstatementnormallyaddsanewlineattheendofitsoutput.Toeliminate
that,addacommaattheend.
Exercises:
1. Printasinglestring.
2. Printthreestringsusingasingleprintstatement.
3. Givenavariablenamecontainingastring,printoutthestringMy name is
"xxxx".,wherexxxxisreplacebythevalueofname.Usethestring
formattingoperator.
Solutions:
1. Wecanprintaliteralstring:
>>>print'Hello,there'
Hello,there
2. Wecanprintliteralsandthevalueofvariables:
>>>description='cute'
>>>print'Iama',description,'kid.'
Iamacutekid.
3. Thestringformattingoperatorgivesmorecontroloverformattingoutput:
>>>name='Alice'
>>>print'Mynameis"%s".'%(name,)
Mynameis"Alice".
3.5.3if:statementexercises
Theifstatementisacompoundstatementthatenablesustoconditionallyexecute
blocksofcode.
Theifstatementalsohasoptionalelif:andelse:clauses.
Theconditioninanif:orelif:clausecanbeanyPythonexpression,inotherwords,
somethingthatreturnsavalue(evenifthatvalueisNone).
Intheconditioninanif:orelif:clause,thefollowingvaluescountas"false":
False
None
Numericzero
Anemptycollection,forexampleanemptylistordictionary
Anemptystring(astringoflengthzero)
Allothervaluescountastrue.
Exercises:
1. Giventhefollowinglist:
>>>bananas=['banana1','banana2','banana3',]
Page148
Printonemessageifitisanemptylistandanothermessgeifitisnot.
2. HereisonewayofdefiningaPythonequivalentofan"enum":
NO_COLOR,RED,GREEN,BLUE=range(4)
Writeanif:statementwhichimplementstheeffectofa"switch"statementin
Python.Printoutauniquemessageforeachcolor.
Solutions:
1. Wecantestforanemptyornonemptylist:
>>>bananas=['banana1','banana2','banana3',]
>>>ifnotbananas:
...print'yes,wehavenobananas'
...else:
...print'yes,wehavebananas'
...
yes,wehavebananas
2. Wecansimulatea"switch"statementusingif:elif: ...:
NO_COLOR,RED,GREEN,BLUE=range(4)
deftest(color):
ifcolor==RED:
print"It'sred."
elifcolor==GREEN:
print"It'sgreen."
elifcolor==BLUE:
print"It'sblue."
defmain():
color=BLUE
test(color)
main()
Which,whenrunprintsoutthefollowing:
It'sblue.
3.5.4for:statementexercises
Thefor:statementisthePythonwaytoiterateoverandprocesstheelementsofa
collectionorotheriterable.
Thebasicformofthefor:statementisthefollowing:
forXinY:
statement
o
o
o
where:
Xissomethingthatcanbeassignedto.ItissomethingtowhichPythoncan
bindavalue.
Page149
Yissomecollectionorotheriterable.
Exercises:
1. Createalistofintegers.Useafor:statementtoprintouteachintegerinthe
list.
2. Createastring.printouteachcharacterinthestring.
Solutions:
1. Thefor:statementcaniterateovertheitemsinalist:
In[13]:a=[11,22,33,]
In[14]:forvalueina:
....:print'value:%d'%value
....:
....:
value:11
value:22
value:33
2. Thefor:statementcaniterateoverthecharactersinastring:
In[16]:b='chocolate'
In[17]:forchr1inb:
....:print'character:%s'%chr1
....:
....:
character:c
character:h
character:o
character:c
character:o
character:l
character:a
character:t
character:e
Notes:
Inthesolution,Iusedthevariablenamechr1ratherthanchrsoasnotto
overwritethenameofthebuiltinfunctionchr().
Whenweneedasequentialindex,wecanusetherange()builtinfunctiontocreatea
listofintegers.And,thexrange()builtinfunctionproducesaninteratorthat
producesasequenceofintegerswithoutcreatingtheentirelist.Toiterateoveralarge
sequenceofintegers,usexrange()insteadofrange().
Exercises:
1. Printouttheintegersfrom0to5insequence.
2. Computethesumofalltheintegersfrom0to99999.
3. Giventhefollowinggeneratorfunction:
importurllib
Urls=[
'http://yahoo.com',
'http://python.org',
'http://gimp.org',#TheGNUimagemanipulation
program
]
Page150
defwalk(url_list):
forurlinurl_list:
f=urllib.urlopen(url)
stuff=f.read()
f.close()
yieldstuff
Writeafor:statementthatusesthisiteratorgeneratortoprintthelengthsof
thecontentateachoftheWebpagesinthatlist.
Solutions:
1. Therange()builtinfunctiongivesusasequencetoiterateover:
In[5]:foridxinrange(6):
...:print'idx:%d'%idx
...:
...:
idx:0
idx:1
idx:2
idx:3
idx:4
idx:5
2. Sincethatsequenceisabitlarge,we'llusexrange()insteadofrange():
In[8]:count=0
In[9]:forninxrange(100000):
...:count+=n
...:
...:
In[10]:count
Out[10]:4999950000
3. Thefor:statementenablesustoiterateoveriterablesaswellascollections:
importurllib
Urls=[
'http://yahoo.com',
program
]
defwalk(url_list):
forurlinurl_list:
stuff=f.read()
f.close()
yieldstuff
deftest():
forurlinwalk(Urls):
print'length:%d'%(len(url),)
test()
WhenIranthisscript,itprintsthefollowing:
Page151
length:9562
length:16341
length:12343
Ifyouneedanindexwhileiteratingoverasequence,considerusingtheenumerate()
builtinfunction.
Exercises:
1. Giventhefollowingtwolistsofintegersofthesamelength:
a=[1,2,3,4,5]
b=[100,200,300,400,500]
Addthevaluesinthefirstlisttothecorrespondingvaluesinthesecondlist.
Solutions:
1. Theenumerate()builtinfunctiongivesusanindexandvaluesfroma
sequence.Sinceenumerate()givesusaninteratorthatproducesasequence
oftwotuples,wecanunpackthosetuplesintoindexandvaluevariablesinthe
headerlineoftheforstatement:
In[13]:a=[1,2,3,4,5]
In[14]:b=[100,200,300,400,500]
In[15]:
In[16]:foridx,valueinenumerate(a):
....:b[idx]+=value
....:
....:
In[17]:b
Out[17]:[101,202,303,404,505]
3.5.5while:statementexercises
Awhile:statementexecutesablockofcoderepeatedlyaslongasaconditionistrue.
Hereisatemplateforthewhile:statement:
whilecondition:
statement
o
o
o
Where:
conditionisanexpression.Theexpressionissomethingthatreturnsavalue
whichcanbeinterpretedastrueorfalse.
Exercises:
1. Writeawhile:loopthatdoublesallthevaluesinalistofintegers.
Solutions:
1. Awhile:loopwithanindexvariablecanbeusedtomodifyeachelementofa
list:
deftest_while():
numbers=[11,22,33,44,]
Page152
print'before:%s'%(numbers,)
idx=0
whileidx<len(numbers):
numbers[idx]*=2
idx+=1
print'after:%s'%(numbers,)
But,noticethatthistaskiseasierusingthefor:statementandthebuiltin
enumerate()function:
deftest_for():
numbers=[11,22,33,44,]
foridx,iteminenumerate(numbers):
numbers[idx]*=2
3.5.6breakandcontinuestatements
Thecontinuestatementskipstheremainderofthestatementsinthebodyofaloop
andstartsimmediatelyatthetopoftheloopagain.
Abreakstatementinthebodyofaloopterminatestheloop.Itexitsfromthe
immediatelycontainingloop.
breakandcontinuecanbeusedinbothfor:andwhile:statements.
Exercises:
1. Writeafor:loopthattakesalistofintegersandtripleseachintegerthatis
even.Usethecontinuestatement.
2. Writealoopthattakesalistofintegersandcomputesthesumofallthe
integersupuntilazeroisfoundinthelist.Usethebreakstatement.
Solutions:
1. Thecontinuestatementenablesusto"skip"itemsthatsatisfyaconditionor
test:
deftest():
numbers=[11,22,33,44,55,66,]
foridx,iteminenumerate(numbers):
ifitem%2!=0:
continue
numbers[idx]*=3
test()
2. Thebreakstatementenablesustoexitfromaloopwhenwefindazero:
deftest():
numbers=[11,22,33,0,44,55,66,]
print'numbers:%s'%(numbers,)
sum=0
foriteminnumbers:
ifitem==0:
break
Page153
sum+=item
print'sum:%d'%(sum,)
test()
3.5.7Exceptionsandthetry:except:andraisestatements
Thetry:except:statementenablesustocatchanexceptionthatisthrownfromwithin
ablockofcode,orfromcodecalledfromanydepthwithingthatblock.
Theraisestatementenablesustothrowanexception.
Anexceptionisaclassoraninstanceofanexceptionclass.Ifanexceptionisnot
caught,itresultsinatracebackandterminationoftheprogram.
Thereisasetofstandardexceptions.Youcanlearnaboutthemhere:Builtin
Exceptionshttp://docs.python.org/lib/moduleexceptions.html.
Youcandefineyourownexceptionclasses.Todoso,createanemptysubclassofthe
classException.Definingyourownexceptionwillenableyou(orothers)tothrow
andthencatchthatspecificexceptiontypewhileignoreothersexceptions.
Exercises:
1. Writeatry:except:statementthatattemptstoopenafileforreadingand
catchestheexceptionthrownwhenthefiledoesnotexist.
Question:Howdoyoufindoutthenameoftheexceptionthatisthrownforan
input/outputerrorsuchasthefailuretoopenafile?
2. Defineanexceptionclass.Thenwriteatry:except:statementinwhichyou
throwandcatchthatspecificexception.
3. Defineanexceptionclassanduseittoimplementamultilevelbreakfroman
innerloop,bypassinganouterloop.
Solutions:
1. UsethePythoninteractiveinterpretertolearntheexceptiontypethrownwhen
aI/Oerroroccurs.Example:
>>>infile=open('xx_nothing__yy.txt','r')
File"<stdin>",line1,in<module>
IOError:[Errno2]Nosuchfileordirectory:
'xx_nothing__yy.txt'
>>>
Inthiscase,theexceptiontypeisIOError.
Now,writeatry:except:blockwhichcatchesthatexception:
deftest():
infilename='nothing_noplace.txt'
try:
forlineininfile:
printline
exceptIOError,exp:
print'cannotopenfile"%s"'%infilename
Page154
test()
2. WedefineaexceptionclassasasubclassofclassException,thenthrowit
(withtheraisestatement)andcatchit(withatry:except:statement):
classSizeError(Exception):
pass
deftest_exception(size):
try:
ifsize<=0:
raiseSizeError,'sizemustbegreaterthan
zero'
#Produceadifferenterrortoshowthatitwillnot
becaught.
x=y
exceptSizeError,exp:
print'%s'%(exp,)
print'goodbye'
deftest():
test_exception(1)
print''*40
test_exception(1)
test()
Whenwerunthisscript,itproducesthefollowingoutput:
$pythonworkbook027.py
sizemustbegreaterthanzero
goodbye
File"workbook027.py",line20,in<module>
test()
File"workbook027.py",line18,intest
test_exception(1)
File"workbook027.py",line10,intest_exception
x=y
NameError:globalname'y'isnotdefined
Notes:
Ourexcept:clausecaughttheSizeError,butallowedtheNameErrorto
beuncaught.
3. WedefineasubclassofofclassException,thenraiseitinaninnerloopand
catchitoutsideofanouterloop:
classBreakException1(Exception):
pass
deftest():
a=[11,22,33,44,55,66,]
b=[111,222,333,444,555,666,]
try:
forxina:
print'outerx:%d'%x
foryinb:
ifx>22andy>444:
raiseBreakException1('leavinginner
Page155
loop')
print'innery:%d'%y
print'outerafter'
print''*40
exceptBreakException1,exp:
print'outofloopexp:%s'%exp
test()
Hereiswhatthisprintsoutwhenrun:
outerx:11
innery:111
innery:222
innery:333
innery:444
innery:555
innery:666
outerafter
outerx:22
innery:111
innery:222
innery:333
innery:444
innery:555
innery:666
outerafter
outerx:33
innery:111
innery:222
innery:333
innery:444
outofloopexp:leavinginnerloop
3.6Functions
Afunctionhasthesecharacteristics:
Itgroupsablockofcodetogethersothatwecancallitbyname.
Itenablesustopassvaluesintothethefunctionwhenwecallit.
Itcanreturnsavalue(evenifNone).
Whenafunctioniscalled,ithasitsownnamespace.Variablesinthefunction
arelocaltothefunction(anddisappearwhenthefunctionexits).
Afunctionisdefinedwiththedef:statement.Hereisasimpleexample/template:
deffunction_name(arg1,arg2):
local_var1=arg1+1
local_var2=arg2*2
returnlocal_var1+local_var2
And,hereisanexampleofcallingthisfunction:
result=function_name(1,2)
Hereareafewnotesofexplanation:
Page156
Theabovedefinesafunctionwhosenameisfunction_name.
Thefunctionfunction_namehastwoarguments.Thatmeansthatwecanand
mustpassinexactlytwovalueswhenwecallit.
Thisfunctionhastwolocalvariables,local_var1andlocal_var2.These
variablesarelocalinthesensethatafterwecallthisfunction,thesetwo
variablesarenotavailableinthelocationofthecaller.
Whenwecallthisfunction,itreturnsonevalue,specificallythesumof
local_var1andlocal_var2.
Exercises:
1. Writeafunctionthattakesalistofintegersasanargument,andreturnsthe
sumoftheintegersinthatlist.
Solutions:
1. Thereturnstatementenablesustoreturnavaluefromafunction:
deflist_sum(values):
sum=0
forvalueinvalues:
sum+=value
returnsum
deftest():
a=[11,22,33,44,]
printlist_sum(a)
test()
3.6.1Optionalargumentsanddefaultvalues
Youcanprovideadefaultvalueforanargumenttoafunction.
Ifyoudo,thatargumentisoptional(whenthefunctioniscalled).
Hereareafewthingstolearnaboutoptionalarguments:
Provideadefaultvaluewithanequalsignandavalue.Example:
defsample_func(arg1,arg2,arg3='empty',arg4=0):
Allparameterswithdefaultvaluesmustbeafter(totherightof)normal
parameters.
Donotuseamutableobjectasadefaultvalue.Becausethedef:statementis
notevaluatedonlyonceandnoteachtimethefunctioniscalled,theobject
maybesharedacrossmultiplecallstothefunction.Donotdothis:
defsample_func(arg1,arg2=[]):
Instead,dothis:
defsample_func(arg1,arg2=None):
ifarg2isNone:
arg2=[]
Hereisanexamplethatillustrateshowthismightgowrong:
Page157
defadder(a,b=[]):
b.append(a)
returnb
deftest():
printadder('aaa')
printadder('bbb')
printadder('ccc')
test()
Which,whenexecuted,displaysthefollowing:
['aaa']
['aaa','bbb']
['aaa','bbb','ccc']
Exercises:
1. Writeafunctionthatwritesastringtoafile.Thefunctiontakestwo
arguments:(1)afilethatisopenforoutputand(2)astring.Givethesecond
argument(thestring)adefaultvaluesothatwhenthesecondargumentis
omitted,anempty,blanklineiswrittentothefile.
2. Writeafunctionthattakesthefollowingarguments:(1)aname,(2)avalue,
and(3)andoptionaldictionary.Thefunctionaddsthevaluetothedictionary
usingthenameasakeyinthedictionary.
Solutions:
1. Wecanpassafileaswewouldanyotherobject.And,wecanuseanewline
characterasadefaultparametervalue:
importsys
defwriter(outfile,msg='\n'):
outfile.write(msg)
deftest():
writer(sys.stdout,'aaaaa\n')
writer(sys.stdout)
writer(sys.stdout,'bbbbb\n')
test()
Whenrunfromthecommandline,thisprintsoutthefollowing:
aaaaa
bbbbb
2. Inthissolutionwearecarefulnottouseamutableobjectasadefaultvalue:
defadd_to_dict(name,value,dic=None):
ifdicisNone:
dic={}
dic[name]=value
returndic
deftest():
dic1={'albert':'cute',}
Page158
printadd_to_dict('barry','funny',dic1)
printadd_to_dict('charlene','smart',dic1)
printadd_to_dict('darryl','outrageous')
printadd_to_dict('eddie','friendly')
test()
Ifwerunthisscript,wesee:
{'barry':'funny','albert':'cute'}
{'barry':'funny','albert':'cute','charlene':'smart'}
{'darryl':'outrageous'}
{'eddie':'friendly'}
Notes:
It'simportantthatthedefaultvalueforthedictionaryisNoneratherthan
anemptydictionary,forexample({}).Rememberthatthedef:statementis
evaluatedonlyonce,whichresultsinasingledictionary,whichwouldbe
sharedbyallcallersthatdonotprovideadictionaryasanargument.
3.6.2Passingfunctionsasarguments
Afunction,likeanyotherobject,canbepassedasanargumenttoafunction.Thisis
duethethefactthatalmostall(maybeall)objectsinPythonare"firstclassobjects".
Afirstclassobjectisonewhichwecan:
1. Storeinadatastructure(e.g.alist,adictionary,...).
2. Passtoafunction.
3. Returnfromafunction.
Exercises:
1. Writeafunctionthattakesthreearguments:(1)aninputfile,(2)anoutputfile,
and(3)afilterfunction:
Argument1isafileopenedforreading.
Argument2isafileopenedforwriting.
Argument3isafunctionthattakesasingleargument(astring),performsa
transformationonthatstring,andreturnsthetransformedstring.
Theabovefunctionshouldreadeachlineintheinputtextfile,passthatline
throughthefilterfunction,thenwritethat(possibly)transformedlinetothe
outputfile.
Now,writeoneormore"filterfunctions"thatcanbepassedtothefunction
describedabove.
Solutions:
1. Thisscriptaddsorremovescommentcharacterstothelinesofafile:
importsys
deffilter(infile,outfile,filterfunc):
forlineininfile:
line=filterfunc(line)
outfile.write(line)
defadd_comment(line):
Page159
line='##%s'%(line,)
returnline
defremove_comment(line):
ifline.startswith('##'):
line=line[3:]
returnline
defmain():
filter(sys.stdin,sys.stdout,add_comment)
main()
Runningthismightproducesomethinglikethefollowing(noteforMS
Windowsusers:usetypeinsteadofcat):
$cattmp.txt
line1
line2
line3
$cattmp.txt|pythonworkbook005.py
##line1
##line2
##line3
3.6.3Extraargsandkeywordargs
Additionalpositionalargumentspassedtoafunctionthatarenotspecifiedinthe
functiondefinition(thedef:statement``),arecollectedinanargumentprecededbya
singleasterisk.Keywordargumentspassedtoafunctionthatarenotspecifiedinthe
functiondefinitioncanbecollectedinadictionaryandpassedtoanargument
precededbyadoubleasterisk.
Examples:
1. Writeafunctionthattakesonepositionalargument,oneargumentwitha
defaultvalue,andalsoextraargsandkeywordargs.
2. Writeafunctionthatpassesallitsarguments,nomatterhowmany,toacallto
anotherfunction.
Solutions:
1. Weuse*argsand**kwargstocollectextraargumentsandextrakeyword
arguments:
defshow_args(x,y=1,*args,**kwargs):
print''*40
print'x:',x
print'y:',y
print'args:',args
deftest():
show_args(1)
show_args(x=2,y=3)
show_args(y=5,x=4)
show_args(4,5,6,7,8)
Page160
show_args(11,y=44,a=55,b=66)
test()
Runningthisscriptproducesthefollowing:
$pythonworkbook006.py
x:1
y:1
args:()
kwargs:{}
x:2
y:3
args:()
kwargs:{}
x:4
y:5
args:()
kwargs:{}
x:4
y:5
args:(6,7,8)
kwargs:{}
x:11
y:44
args:()
kwargs:{'a':55,'b':66}
Notes:
Thespellingofargsandkwargsisnotfixed,butthe
2. Weuseargsandkwargstocatchandpassonallarguments:
deffunc1(*args,**kwargs):
print'args:%s'%(args,)
print'kwargs:%s'%(kwargs,)
deffunc2(*args,**kwargs):
print'before'
func1(*args,**kwargs)
print'after'
deftest():
func2('aaa','bbb','ccc',arg1='ddd',arg2='eee')
test()
Whenwerunthis,itprintsthefollowing:
before
args:('aaa','bbb','ccc')
kwargs:{'arg1':'ddd','arg2':'eee'}
after
Notes:
Inafunctioncall,the*operatorunrollsalistintoindividualpositional
Page161
arguments,andthe**operatorunrollsadictionaryintoindividualkeyword
arguments.
3.6.3.1Orderofarguments(positional,extra,andkeywordargs)
Inafunctiondefinition,argumentsmustappearinthefollowingorder,fromleftto
right:
1. Positional(normal,plain)arguments
2. Argumentswithdefaultvalues,ifany
3. Extraargumentsparameter(procededbysingleasterisk),ifpresent
4. Keywordargumentsparameter(procededbydoubleasterisk),ifpresent
Inafunctioncall,argumentsmustappearinthefollowingorder,fromlefttoright:
1. Positional(plain)arguments
2. Extraarguments,ifpresent
3. Keywordarguments,ifpresent
3.6.4Functionsandducktypingandpolymorphism
Iftheargumentsandreturnvalueofafunctionsatisfysomedescription,thenwecan
saythatthefunctionispolymorphicwithrespecttothatdescription.
Ifthesomeofthemethodsofanobjectsatisfysomedescription,thenwecansaythat
theobjectispolymorphicwithrespecttothatdescription.
Basically,whatthisdoesistoenableustouseafunctionoranobjectanywherethat
functionsatisfiestherequirementsgivenbyadescription.
Exercises:
1. Implementafunctionthattakestwoarguments:afunctionandanobject.It
appliesthefunctionargumenttotheobject.
2. Implementafunctionthattakestwoarguments:alistoffunctionsandan
object.Itapplieseachfunctioninthelisttotheargument.
Solutions:
1. Wecanpassafunctionasanargumenttoafunction:
deffancy(obj):
print'fancyfancy%sfancyfancy'%(obj,)
defplain(obj):
print'plain%splain'%(obj,)
defshow(func,obj):
func(obj)
defmain():
a={'aa':11,'bb':22,}
show(fancy,a)
show(plain,a)
main()
Page162
2. Wecanalsoputfunctions(functionobjects)inadatastructure(forexample,a
list),andthenpassthatdatastructuretoafunction:
deffancy(obj):
print'fancyfancy%sfancyfancy'%(obj,)
defplain(obj):
print'plain%splain'%(obj,)
Func_list=[fancy,plain,]
defshow(funcs,obj):
forfuncinfuncs:
func(obj)
defmain():
a={'aa':11,'bb':22,}
show(Func_list,a)
main()
NoticethatPythonsupportspolymorphism(withor)withoutinheritance.Thistypeof
polymorphismisenabledbywhatiscalledducktyping.Formoreonthissee:Duck
typinghttp://en.wikipedia.org/wiki/Duck_typingatWikipedia.
3.6.5Recursivefunctions
Arecursivefunctionisafunctionthatcallsitself.
Arecursivefunctionmusthavealimitingcondition,orelseitwillloopendlessly.
Eachrecursivecallconsumesspaceonthefunctioncallstack.Therefore,thenumber
ofrecursionsmusthavesomereasonableupperbound.
Exercises:
1. Writearecursivefunctionthatprintsinformationabouteachnodeinthe
followingtreestructuredatastructure:
Tree={
'name':'animals',
'left_branch':{
'name':'birds',
'left_branch':{
'name':'seedeaters',
'left_branch':{
'name':'housefinch',
'left_branch':None,
'right_branch':None,
},
'right_branch':{
'name':'whitecrownedsparrow',
'left_branch':None,
},
},
'right_branch':{
'name':'insecteaters',
Page163
'left_branch':{
'name':'hermitthrush',
'left_branch':None,
},
'right_branch':{
'name':'blackheadedphoebe',
'left_branch':None,
},
},
},
}
Solutions:
1. Wewritearecursivefunctiontowalkthewholetree.Therecursivefunction
callsitselftoprocesseachchildofanodeinthetree:
Tree={
'name':'animals',
'left_branch':{
'name':'birds',
'left_branch':{
'name':'seedeaters',
'left_branch':{
'name':'housefinch',
'left_branch':None,
},
'right_branch':{
'name':'whitecrownedsparrow',
'left_branch':None,
},
},
'right_branch':{
'name':'insecteaters',
'left_branch':{
'name':'hermitthrush',
'left_branch':None,
},
'right_branch':{
'name':'blackheadedphoebe',
'left_branch':None,
},
},
},
}
Indents=[''*idxforidxinrange(10)]
defwalk_and_show(node,level=0):
ifnodeisNone:
return
print'%sname:%s'%(Indents[level],node['name'],)
Page164
level+=1
walk_and_show(node['left_branch'],level)
walk_and_show(node['right_branch'],level)
deftest():
walk_and_show(Tree)
test()
Notes:
Later,youwilllearnhowtocreateequivalentdatastructuresusingclasses
andOOP(objectorientedprogramming).FormoreonthatseeRecursive
callstomethodsinthisdocument.
3.6.6Generatorsanditerators
The"iteratorprotocol"defineswhataniteratorobjectmustdoinordertobeusablein
an"iteratorcontext"suchasaforstatement.Theiteratorprotocolisdescribedinthe
standardlibraryreference:IteratorTypeshttp://docs.python.org/lib/typeiter.html
Aneasywaytodefineanobjectthatobeystheiteratorprotocolistowriteagenerator
function.Ageneratorfunctionisafunctionthatcontainsoneormoreyield
statements.Ifafunctioncontainsatleastoneyieldstatement,thenthatfunctionwhen
called,returnsgeneratoriterator,whichisanobjectthatobeystheiteratorprotocol,
i.e.it'saniteratorobject.
NotethatinrecentversionsofPython,yieldisanexpression.Thisenablesthe
consumertocommunicatebackwiththeproducer(thegeneratoriterator).Formore
onthis,seePEP:342CoroutinesviaEnhancedGenerators
http://www.python.org/dev/peps/pep0342/.
Exercises:
1. ImplementageneratorfunctionThegeneratorproducedshouldyieldall
valuesfromalist/iterablethatsatisfyapredicate.Itshouldapplythe
transformsbeforereturneachvalue.Thefunctiontakesthesearguments:
1. valuesAlistofvalues.Actually,itcouldbeanyiterable.
2. predicateAfunctionthattakesasingleargument,performsateston
thatvalue,andreturnsTrueorFalse.
3. transforms(optional)Alistoffunctions.Applyeachfunctioninthis
listandreturnstheresultingvalue.So,forexample,ifthefunctionis
calledlikethis:
result=transforms([11,22],p,[f,g])
thentheresultinggeneratormightreturn:
g(f(11))
2. ImplementageneratorfunctionthattakesalistofURLsasitsargumentand
generatesthecontentsofeachWebpage,onebyone(thatis,itproducesa
sequenceofstrings,theHTMLpagecontents).
Solutions:
Page165
1. Hereistheimplementationofafunctionwhichcontainsyield,and,therefore,
producesagenerator:
"""
filter_and_transform
filter_and_transform(content,test_func,transforms=None)
Returnageneratorthatreturnsitemsfromcontentafter
applying
thefunctionsintransformsiftheitemsatisfies
test_func.
Arguments:
1.``values`Àlistofvalues
2.``predicate`Àfunctionthattakesasingle
argument,
performsatestonthatvalue,andreturnsTrueor
False.
3.``transforms``(optional)Alistoffunctions.
Applyeach
functioninthislistandreturnstheresultingvalue.
So,
forexample,ifthefunctioniscalledlikethis::
result=filter_and_transforms([11,22],p,[f,g])
thentheresultinggeneratormightreturn::
g(f(11))
"""
deffilter_and_transform(content,test_func,
transforms=None):
forxincontent:
iftest_func(x):
iftransformsisNone:
yieldx
elifisiterable(transforms):
forfuncintransforms:
x=func(x)
yieldx
else:
yieldtransforms(x)
defisiterable(x):
flag=True
try:
x=iter(x)
exceptTypeError,exp:
flag=False
returnflag
defiseven(n):
returnn%2==0
Page166
deff(n):
returnn*2
defg(n):
returnn**2
deftest():
data1=[11,22,33,44,55,66,77,]
forvalinfilter_and_transform(data1,iseven,f):
print'val:%d'%(val,)
print''*40
forvalinfilter_and_transform(data1,iseven,[f,g]):
print''*40
forvalinfilter_and_transform(data1,iseven):
test()
Notes:
Becausefunctionfilter_and_transformcontainsyield,whencalled,it
returnsaniteratorobject,whichwecanuseinaforstatement.
Thesecondparameteroffunctionfilter_and_transformtakesany
functionwhichtakesasingleargumentandreturnsTrueorFalse.Thisis
anexampleofpolymorphismand"ducktyping"(seeDuckTyping
http://en.wikipedia.org/wiki/Duck_typing).Ananalogousclaimcanbe
madeaboutthethirdparameter.
2. Thefollowingfunctionusestheurllibmoduleandtheyieldfunctionto
generatethecontentsofasequenceofWebpages:
importurllib
Urls=[
'http://yahoo.com',
program
]
defwalk(url_list):
forurlinurl_list:
stuff=f.read()
f.close()
yieldstuff
deftest():
forxinwalk(Urls):
print'length:%d'%(len(x),)
test()
WhenIrunthis,Isee:
$pythongenerator_example.py
length:9554
Page167
length:16748
length:11487
3.7Objectorientedprogrammingandclasses
ClassesprovidePython'swaytodefinenewdatatypesandtodoOOP(object
orientedprogramming).
Ifyouhavemadeitthisfar,youhavealreadyusedlotsofobjects.Youhavebeena
"consumer"ofobjectsandtheirservices.Now,youwilllearnhowtodefineand
implementnewkindsofobjects.Youwillbecomea"producer"ofobjects.Youwill
definenewclassesandyouwillimplementthecapabilities(methods)ofeachnew
class.
Aclassisdefinedwiththeclassstatement.Thefirstlineofaclassstatementisa
headerforacompoundstatement(ithasacolonattheend),anditspecifiesthename
oftheclassbeingdefinedandan(option)superclass.Thebodyoftheclassstatement
containsstatements,importantly,defstatementsthatdefinethemethodsthatcanbe
calledoninstancesoftheobjectsimplementedbythisclass.
Exercises:
1. Defineaclasswithonemethodshow.Thatmethodshouldprintout"Hello".
Then,createaninstanceofyourclass,andcalltheshowmethod.
Solutions:
1. Asimpleinstancemethodcanhavetheselfparameterandnoothers:
classDemo(object):
defshow(self):
print'hello'
deftest():
a=Demo()
a.show()
test()
Notes:
Noticethatweuseobjectasasuperclass,becausewewanttodefinean
"newstyle"classandbecausethereisnootherclassthatwewantasa
superclass.Seethefollowingformoreinformationonnewstyleclasses:
NewstyleClasseshttp://www.python.org/doc/newstyle/.
InPython,wecreateaninstanceofaclassbycallingtheclass,thatis,we
applythefunctioncalloperator(parentheses)totheclass.
3.7.1Theconstructor
Aclasscandefinemethodswithspecialnames.Youhaveseemsomeofthesebefore.
Thesenamesbeginandendwithadoubleunderscore.
Oneimportantspecialnameis__init__.It'stheconstructorforaclass.Itiscalled
eachtimeaninstanceoftheclassiscreated.Implementingthismethodinaclass
Page168
givesusachancetoinitializeeachinstanceofourclass.
Exercises:
1. ImplementaclassnamedPlantthathasaconstructorwhichinitializestwo
instancevariables:nameandsize.Also,inthisclass,implementamethod
namedshowthatprintsoutthevaluesoftheseinstancevariables.Create
severalinstancesofyourclassand"show"them.
2. ImplementaclassnameNodethathastwoinstancevariables:dataand
children,wheredataisany,arbitraryobjectandchildrenisalistofchild
Nodes.Alsoimplementamethodnamedshowthatrecursivelydisplaysthe
nodesina"tree".Createaninstanceofyourclassthatcontainsseveralchild
instancesofyourclass.Calltheshowmethodontheroot(topmost)objectto
showthetree.
Solutions:
1. Theconstructorforaclassisamethodwiththespecialname__init__:
classPlant(object):
def__init__(self,name,size):
self.name=name
self.size=size
defshow(self):
print'name:"%s"size:%d'%(self.name,
self.size,)
deftest():
p1=Plant('Eggplant',25)
p2=Plant('Tomato',36)
plants=[p1,p2,]
forplantinplants:
plant.show()
test()
Notes:
Ourconstructortakestwoarguments:nameandsize.Itsavesthosetwo
valuesasinstancevariables,thatisinattributesoftheinstance.
Theshow()methodprintsoutthevalueofthosetwoinstancevariables.
2. Itisagoodideatoinitializeallinstancevariablesintheconstructor.That
enablessomeonereadingourcodetolearnaboutalltheinstancevariablesofa
classbylookinginasinglelocation:
#simple_node.py
Indents=[''*nforninrange(10)]
classNode(object):
def__init__(self,name=None,children=None):
self.name=name
ifchildrenisNone:
self.children=[]
else:
defshow_name(self,indent):
print'%sname:"%s"'%(Indents[indent],
Page169
self.name,)
defshow(self,indent=0):
self.show_name(indent)
indent+=1
child.show(indent)
deftest():
n1=Node('N1')
n2=Node('N2')
n3=Node('N3')
n4=Node('N4')
n5=Node('N5',[n1,n2,])
n6=Node('N6',[n3,n4,])
n7=Node('N7',[n5,n6,])
n7.show()
test()
Notes:
Noticethatwedonotusetheconstructorforalist([])asadefaultvalue
forthechildrenparameteroftheconstructor.Alistismutableandwould
becreatedonlyonce(whentheclassstatementisexecuted)andwouldbe
shared.
3.7.2InheritanceImplementingasubclass
Asubclassextendsorspecializesasuperclassbyaddingadditionalmethodstothe
superclassandbyoverridingmethods(withthesamename)thatalreadyexistinthe
superclass.
Exercises:
1. ExtendyourNodeexerciseabovebyaddingtwoadditionalsubclassesofthe
Nodeclass,onenamedPlantandtheothernamedAnimal.ThePlantclass
alsohasaheightinstancevariableandtheAnimalclassalsohasacolor
instancevariable.
Solutions:
1. WecanimportourpreviousNodescript,thenimplementclassesthathavethe
Nodeclassasasuperclass:
fromsimple_nodeimportNode,Indents
classPlant(Node):
def__init__(self,name,height=1,children=None):
Node.__init__(self,name,children)
self.height=height
print'%sheight:%s'%(Indents[indent],
self.height,)
indent+=1
child.show(indent)
Page170
classAnimal(Node):
def__init__(self,name,color='nocolor',
children=None):
Node.__init__(self,name,children)
self.color=color
print'%scolor:"%s"'%(Indents[indent],
self.color,)
indent+=1
child.show(indent)
deftest():
n1=Animal('scrubjay','grayblue')
n2=Animal('raven','black')
n3=Animal('americankestrel','brown')
n4=Animal('redshoulderedhawk','brownandgray')
n5=Animal('corvid','none',[n1,n2,])
n6=Animal('raptor',children=[n3,n4,])
n7a=Animal('bird',children=[n5,n6,])
n1=Plant('valleyoak',50)
n2=Plant('canyonliveoak',40)
n3=Plant('jefferypine',120)
n4=Plant('ponderosapine',140)
n5=Plant('oak',children=[n1,n2,])
n6=Plant('conifer',children=[n3,n4,])
n7b=Plant('tree',children=[n5,n6,])
n8=Node('birdsandtrees',[n7a,n7b,])
n8.show()
test()
Notes:
TheshowmethodinclassPlantcallstheshow_namemethodinits
superclassusingself.show_name(...).Pythonsearchesuptheinheritance
treetofindtheshow_namemethodinclassNode.
Theconstructor(__init__)inclassesPlantandAnimaleachcallthe
constructorinthesuperclassbyusingthenameofthesuperclass.Whythe
difference?Because,if(inthePlantclass,forexample)itused
self.__init__(...)itwouldbecallingthe__init__inthePlantclass,itself.
So,itbypassesitselfbyreferencingtheconstructorinthesuperclass
directly.
Thisexercisealsodemonstrates"polymorphism"Theshowmethodis
calledanumberoftimes,butwhichimplementationexecutesdependson
whichinstanceitiscalledon.Callingontheshowmethodonaninstance
ofclassPlantresultsinacalltoPlant.show.Callingtheshowmethodon
aninstanceofclassAnimalresultsinacalltoAnimal.show.Andsoon.It
isimportantthateachshowmethodtakesthecorrectnumberofarguments.
3.7.3Classesandpolymorphism
Pythonalsosupportsclassbasedpolymorphism,whichwas,bytheway,
Page171
demonstratedinthepreviousexample.
Exercises:
1. Writethreeclasses,eachofwhichimplementashow()methodthattakesone
argument,astring.Theshowmethodshouldprintoutthenameoftheclass
andthemessage.Thencreatealistofinstancesandcalltheshow()methodon
eachobjectinthelist.
Solution:
1. Weimplementthreesimpleclassesandthencreatealistofinstancesofthese
classes:
classA(object):
defshow(self,msg):
print'classAmsg:"%s"'%(msg,)
classB(object):
defshow(self,msg):
print'classBmsg:"%s"'%(msg,)
classC(object):
defshow(self,msg):
print'classCmsg:"%s"'%(msg,)
deftest():
objs=[A(),B(),C(),A(),]
foridx,objinenumerate(objs):
msg='message#%d'%(idx+1,)
obj.show(msg)
test()
Notes:
Wecancalltheshow()methodinanyobjectinthelistobjsaslongaswe
passinasingleparameter,thatis,aslongasweobeytherequirementsof
ducktyping.Wecandothisbecauseallobjectsinthatlistimplementa
show()method.
Inastaticallytypedlanguage,thatisalanguagewherethetypeis(also)
presentinthevariable,alltheinstancesinexamplewouldhavetodescend
fromacommonsuperclassandthatsuperclasswouldhavetoimplementa
show()method.Pythondoesnotimposethisrestriction.And,because
variablesarenotnottypedinPython,perhapsthatwouldnoteven
possible.
Noticethatthisexampleofpolymorphismworkseventhoughthesethree
classes(A,B,andC)arenotrelated(forexample,inaclasshierarchy).All
thatisrequiredforpolymorphismtoworkinPythonisforthemethod
namestobethesameandtheargumentstobecompatible.
3.7.4Recursivecallstomethods
Amethodinaclasscanrecusivelycallitself.Thisisverysimilartothewayinwhich
weimplementedrecursivefunctionssee:Recursivefunctions.
Page172
Exercises:
1. ReimplementthebinarytreeofanimalsandbirdsdescribedinRecursive
functions,butthistime,useaclasstorepresenteachnodeinthetree.
2. Solvethesameproblem,butthistimeimplementatreeinwhicheachnode
canhaveanynumberofchildren(ratherthanexactly2children).
Solutions:
1. Weimplementaclasswiththreeinstancevariables:(1)name,(2)leftbranch,
and(3)rightbranch.Then,weimplementashow()methodthatdisplaysthe
nameandcallsitselftoshowthechildrenineachsubtree:
Indents=[''*idxforidxinrange(10)]
classAnimalNode(object):
def__init__(self,name,left_branch=None,
right_branch=None):
self.name=name
self.left_branch=left_branch
self.right_branch=right_branch
defshow(self,level=0):
print'%sname:%s'%(Indents[level],self.name,)
level+=1
ifself.left_branchisnotNone:
self.left_branch.show(level)
ifself.right_branchisnotNone:
self.right_branch.show(level)
Tree=AnimalNode('animals',
AnimalNode('birds',
AnimalNode('seedeaters',
AnimalNode('housefinch'),
AnimalNode('whitecrownedsparrow'),
),
AnimalNode('insecteaters',
AnimalNode('hermitthrush'),
AnimalNode('blackheadedphoebe'),
),
),
None,
)
deftest():
Tree.show()
test()
2. Insteadofusingaleftbranchandarightbranch,inthissolutionweusealist
torepresentthechildrenofanode:
classAnimalNode(object):
def__init__(self,data,children=None):
self.data=data
ifchildrenisNone:
self.children=[]
else:
Page173
defshow(self,level=''):
print'%sdata:%s'%(level,self.data,)
level+=''
child.show(level)
Tree=AnimalNode('animals',[
AnimalNode('birds',[
AnimalNode('seedeaters',[
AnimalNode('housefinch'),
AnimalNode('whitecrownedsparrow'),
AnimalNode('lessergoldfinch'),
]),
AnimalNode('insecteaters',[
AnimalNode('hermitthrush'),
AnimalNode('blackheadedphoebe'),
]),
])
])
deftest():
Tree.show()
test()
Notes:
Werepresentthechildrenofanodeasalist.Eachnode"hasa"listof
children.
Noticethatbecausealistismutable,wedonotusealistconstructor([])in
theinitializerofthemethodheader.Instead,weuseNone,thenconstruct
anemptylistinthebodyofthemethodifnecessary.SeesectionOptional
argumentsanddefaultvaluesformoreonthis.
We(recursively)calltheshowmethodforeachnodeinthechildrenlist.
Sinceanodewhichhasnochildren(aleafnode)willhaveanempty
childrenlist,thisprovidesalimitconditionforourrecursion.
3.7.5Classvariables,classmethods,andstaticmethods
Aclassvariableisonewhosesinglevalueissharedbyallinstancesoftheclassand,
infact,issharedbyallwhohaveaccesstotheclass(object).
"Normal"methodsareinstancemethods.Aninstancemethodreceivestheinstanceas
itsfirstargument.Ainstancemethodisdefinedbyusingthedefstatementinthe
bodyofaclassstatement.
Aclassmethodreceivestheclassasitsfirstargument.Aclassmethodisdefinedby
defininganormal/instancemethod,thenusingtheclassmethodbuiltinfunction.For
example:
classASimpleClass(object):
description='asimpleclass'
defshow_class(cls,msg):
Page174
print'%s:%s'%(cls.description,msg,)
show_class=classmethod(show_class)
Astaticmethoddoesnotreceiveanythingspecialasitsfirstargument.Astatic
methodisdefinedbydefininganormal/instancemethod,thenusingthe
staticmethodbuiltinfunction.Forexample:
classASimpleClass(object):
description='asimpleclass'
defshow_class(msg):
print'%s:%s'%(ASimpleClass.description,msg,)
show_class=staticmethod(show_class)
Ineffect,bothclassmethodsandstaticmethodsaredefinedbycreatinganormal
(instance)method,thencreatingawrapperobject(aclassmethodorstaticmethod)
usingtheclassmethodorstaticmethodbuiltinfunction.
Exercises:
1. Implementaclassthatkeepsarunningtotalofthenumberofinstances
created.
2. Implementanothersolutiontothesameproblem(aclassthatkeepsarunning
totalofthenumberofinstances),butthistimeuseastaticmethodinsteadofa
classmethod.
Solutions:
1. Weuseaclassvariablenamedinstance_count,ratherthananinstance
variable,tokeeparunningtotalofinstances.Then,weincrementthatvariable
eachtimeaninstanceiscreated:
classCountInstances(object):
instance_count=0
def__init__(self,name='noname'):
self.name=name
CountInstances.instance_count+=1
defshow(self):
print'name:"%s"'%(self.name,)
defshow_instance_count(cls):
print'instancecount:%d'%(cls.instance_count,)
show_instance_count=classmethod(show_instance_count)
deftest():
instances=[]
instances.append(CountInstances('apple'))
instances.append(CountInstances('banana'))
instances.append(CountInstances('cherry'))
instances.append(CountInstances())
forinstanceininstances:
instance.show()
CountInstances.show_instance_count()
Page175
test()
Notes:
Whenwerunthisscript,itprintsoutthefollowing:
name:"apple"
name:"banana"
name:"cherry"
name:"noname"
instancecount:4
Thecalltotheclassmethodbuiltinfunctioneffectivelywrapsthe
show_instance_countmethodinaclassmethod,thatis,inamethodthat
takesaclassobjectasitsfirstargumentratherthananinstanceobject.To
readmoreaboutclassmethod,gotoBuiltinFunctions
http://docs.python.org/lib/builtinfuncs.htmlandsearchfor
"classmethod".
2. Astaticmethodtakesneitheraninstance(self)noraclassasitsfirst
paramenter.And,staticmethodiscreatedwiththestaticmethod()builtin
function(ratherthanwiththeclassmethod()builtin):
instance_count=0
self.name=name
defshow(self):
defshow_instance_count():
print'instancecount:%d'%(
CountInstances.instance_count,)
show_instance_count=staticmethod(show_instance_count)
deftest():
instances=[]
instance.show()
test()
3.7.5.1Decoratorsforclassmethodandstaticmethod
Adecoratorenablesustodowhatwedidinthepreviousexamplewithasomewhat
simplersyntax.
Forsimplecases,thedecoratorsyntaxenablesustodothis:
Page176
@functionwrapper
defmethod1(self):
o
o
o
insteadofthis:
defmethod1(self):
o
o
o
method1=functionwrapper(method1)
So,wecanwritethis:
@classmethod
defmethod1(self):
o
o
o
insteadofthis:
defmethod1(self):
o
o
o
method1=classmethod(method1)
Exercises:
1. ImplementtheCountInstancesexampleabove,butuseadecoratorrather
thantheexplicitcalltoclassmethod.
Solutions:
1. Adecoratorisaneasierandcleanerwaytodefineaclassmethod(orastatic
method):
instance_count=0
self.name=name
defshow(self):
@classmethod
defshow_instance_count(cls):
print'instancecount:%d'%(cls.instance_count,)
#Notethatthefollowinglinehasbeenreplacedby
#theclassmethoddecorator,above.
#show_instance_count=classmethod(show_instance_count)
deftest():
instances=[]
Page177
instance.show()
test()
3.8AdditionalandAdvancedTopics
3.8.1Decoratorsandhowtoimplementthem
Decoratorscanbeusedto"wrap"afunctionwithanotherfunction.
Whenimplementingadecorator,itishelpfultorememberthatthefollowing
decoratorapplication:
@dec
deffunc(arg1,arg2):
pass
isequivalentto:
deffunc(arg1,arg2):
pass
func=dec(func)
Therefore,toimplementadecorator,wewriteafunctionthatreturnsafunction
object,sincewereplacethevalueoriginallyboundtothefunctionwiththisnew
functionobject.Itmaybehelpfultotaketheviewthatwearecreatingafunctionthat
isawrapperfortheoriginalfunction.
Exercises:
1. Writeadecoratorthatwritesamessagebeforeandafterexecutingafunction.
Solutions:
1. Afunctionthatcontainsandreturnsaninnerfunctioncanbeusedtowrapa
function:
deftrace(func):
definner(*args,**kwargs):
print'>>'
func(*args,**kwargs)
print'<<'
returninner
@trace
deffunc1(x,y):
print'x:',x,'y:',y
func2((x,y))
@trace
Page178
deffunc2(content):
print'content:',content
deftest():
func1('aa','bb')
test()
Notes:
Yourinnerfunctioncanuse*argsand**kwargstoenableittocall
functionswithanynumberofarguments.
3.8.1.1Decoratorswitharguments
Decoratorscanalsotakearguments.
Thefollowingdecoratorwitharguments:
@dec(argA,argB)
deffunc(arg1,arg2):
pass
isequivalentto:
deffunc(arg1,arg2):
pass
func=dec(argA,argB)(func)
Becausethedecorator'sargumentsarepassedtotheresultofcallingthedecoratoron
thedecoratedfunction,youmayfinditusefultoimplementadecoratorwith
argumentsusingafunctioninsideafunctioninsideafunction.
Exercises:
1. Writeandtestadecoratorthattakesoneargument.Thedecoratorprintsa
messagealongwiththevalueoftheargumentbeforeandafterenteringthe
decoratedfunction.
Solutions:
1. Implementthisdecoratorthattakesargumentswithafunctioncontaininga
nestedfunctionwhichinturncontainsanestedfunction:
deftrace(msg):
definner1(func):
definner2(*args,**kwargs):
print'>>[%s]'%(msg,)
retval=func(*args,**kwargs)
print'<<[%s]'%(msg,)
returnretval
returninner2
returninner1
@trace('tracingfunc1')
deffunc1(x,y):
print'x:',x,'y:',y
result=func2((x,y))
returnresult
Page179
deffunc2(content):
returncontent*3
deftest():
result=func1('aa','bb')
test()
3.8.1.2Stackeddecorators
Decoratorscanbe"stacked".
Thefollowingstackeddecorators:
@dec2
@dec1
deffunc(arg1,arg2,...):
pass
areequivalentto:
deffunc(arg1,arg2,...):
pass
func=dec2(dec1(func))
Exercises:
1. Implementadecorator(asabove)thattracescallstoadecoratedfunction.
Then"stack"thatwithanotherdecoratorthatprintsahorizontallineofdashes
beforeandaftercallingthefunction.
2. Modifyyoursolutiontotheaboveexercisesothatthedecoratorthatprintsthe
horizontallinetakesoneargument:acharacter(orcharacters)thatcanbe
repeatedtoproduceahorizontalline/separator.
Solutions:
1. Reuseyourtracingfunctionfromthepreviousexercise,thenwriteasimple
decoratorthatprintsarowofdashes:
deftrace(msg):
definner1(func):
returnretval
returninner2
returninner1
defhorizontal_line(func):
definner(*args,**kwargs):
print''*50
print''*50
returnretval
Page180
returninner
deffunc1(x,y):
print'x:',x,'y:',y
result=func2((x,y))
returnresult
@horizontal_line
deffunc2(content):
returncontent*3
deftest():
test()
2. Onceagain,adecoratorwithargumentscanbeimplementedwithafunction
nestedinsideafunctionwhichisnestedinsideafunction.Thisremainsthe
samewhetherthedecoratorisusedasastackeddecoratorornot.Hereisa
solution:
deftrace(msg):
definner1(func):
returnretval
returninner2
returninner1
defhorizontal_line(line_chr):
definner1(func):
printline_chr*15
printline_chr*15
returnretval
returninner2
returninner1
deffunc1(x,y):
print'x:',x,'y:',y
result=func2((x,y))
returnresult
@horizontal_line('<**>')
deffunc2(content):
returncontent*3
deftest():
Page181
test()
3.8.1.3Morehelpwithdecorators
Thereismoreaboutdecoratorshere:
Pythonsyntaxandsemantics
http://en.wikipedia.org/wiki/Python_syntax_and_semantics#Decoratorsat
Wikipedia.
PythonDecoratorLibrary
http://wiki.python.org/moin/PythonDecoratorLibraryatthePythonWikihas
lotsofsamplecode.
Kent'sKornerPythonDecoratorshttp://personalpages.tds.net/~kent37/kk/
00001.htmlhashelpfulexplanationsandtowardtheendgivesreferencesto
othersourcesofinformationondecorators.
PEP318DecoratorsforFunctionsandMethods
http://www.python.org/dev/peps/pep0318/istheformalproposaland
specificationforPythondecorators.
3.8.2Iterables
3.8.2.1AfewpreliminariesonIterables
Definition:iterable(adjective)thatwhichcanbeiteratedover.
Agoodtestofwhethersomethingisiterableiswhetheritcanbeusedinafor:
statement.Forexample,ifwecanwritefor item in X:,thenXisiterable.Hereis
anothersimpletest:
defisiterable(x):
try:
y=iter(x)
exceptTypeError,exp:
returnFalse
returnTrue
Somekindsofiterables:
ContainersWecaniterateoverlists,tuples,dictionaries,sets,strings,and
othercontainers.
Somebuiltin(noncontainer)typesExamples:
Atextfileopeninreadmodeisiterable:ititeratesoverthelinesinthefile.
ThexrangetypeSeeXRangeTypehttp://docs.python.org/lib/typesseq
xrange.html.It'susefulwhenyouwantalargesequenceofintegersto
iterateover.
Instancesofclassesthatobeytheiteratorprotocol.Foradescriptionofthe
iteratorprotocol,seeIteratorTypeshttp://docs.python.org/lib/typeiter.html.
Hint:Typedir(obj)andlookfor"__iter__"and"next".
Page182
GeneratorsAnobjectreturnedbyanyfunctionormethodthatcontains
yield.
Exercises:
1. Implementaclasswhoseinstancesareinterable.Theconstructortakesalistof
URLsasitsargument.Aninstanceofthisclass,wheniteratedover,generates
thecontentoftheWebpageatthataddress.
Solutions:
1. Weimplementaclassthathas__iter__()andnext()methods:
importurllib
classWebPages(object):
def__init__(self,urls):
self.urls=urls
self.current_index=0
def__iter__(self):
self.current_index=0
returnself
defnext(self):
ifself.current_index>=len(self.urls):
raiseStopIteration
url=self.urls[self.current_index]
self.current_index+=1
content=f.read()
f.close()
returncontent
deftest():
urls=[
'http://www.python.org',
'http://en.wikipedia.org/',
'http://en.wikipedia.org/wiki/Python_(programming_la
nguage)',
]
pages=WebPages(urls)
forpageinpages:
print'length:%d'%(len(page),)
pages=WebPages(urls)
print''*50
page=pages.next()
page=pages.next()
page=pages.next()
page=pages.next()
test()
3.9ApplicationsandRecipies
Page183
3.9.1XML
Exercises:
1. SAXParseanXMLdocumentwithSAX,thenshowsomeinformation(tag,
attributes,characterdata)foreachelement.
2. MinidomParseanXMLdocumentwithminidom,thenwalktheDOMtree
andshowsomeinformation(tag,attributes,characterdata)foreachelement.
HereisasampleXMLdocumentthatyoucanuseforinput:
<?xmlversion="1.0"?>
<people>
<personid="1"value="abcd"ratio="3.2">
<name>Alberta</name>
<interest>gardening</interest>
<interest>reading</interest>
<category>5</category>
</person>
<personid="2">
<name>Bernardo</name>
<interest>programming</interest>
<category></category>
<agent>
<firstname>Darren</firstname>
<lastname>Diddly</lastname>
</agent>
</person>
<personid="3"value="efgh">
<name>Charlie</name>
<interest>people</interest>
<interest>cats</interest>
<interest>dogs</interest>
<category>8</category>
<promoter>
<firstname>David</firstname>
<lastname>Donaldson</lastname>
<client>
<fullname>ArnoldApplebee</fullname>
<refid>10001</refid>
</client>
</promoter>
<promoter>
<firstname>Edward</firstname>
<lastname>Eddleberry</lastname>
<client>
<fullname>ArnoldApplebee</fullname>
<refid>10001</refid>
</client>
</promoter>
</person>
</people>
3. ElementTreeParseanXMLdocumentwithElementTree,thenwalkthe
DOMtreeandshowsomeinformation(tag,attributes,characterdata)foreach
element.
4. lxmlParseanXMLdocumentwithlxml,thenwalktheDOMtreeandshow
someinformation(tag,attributes,characterdata)foreachelement.
5. ModifydocumentwithElementTreeUseElementTreetoreadadocument,
Page184
thenmodifythetree.Showthecontentsofthetree,andthenwriteoutthe
modifieddocument.
6. XPathlxmlsupportsXPath.UsetheXPathsupportinlxmltoaddresseach
ofthefollowingintheaboveXMLinstancedocument:
Thetextinallthenameelements
Thevaluesofalltheidattributes
Solutions:
1. WecanusetheSAXsupportinthePythonstandardlibrary:
"""
ParseandXMLwithSAX.Displayinfoabouteachelement.
Usage:
pythontest_sax.pyinfilename
Examples:
pythontest_sax.pypeople.xml
"""
importsys
fromxml.saximportmake_parser,handler
classTestHandler(handler.ContentHandler):
def__init__(self):
self.level=0
defshow_with_level(self,value):
print'%s%s'%(''*self.level,value,)
defstartDocument(self):
self.show_with_level('Documentstart')
self.level+=1
defendDocument(self):
self.level=1
self.show_with_level('Documentend')
defstartElement(self,name,attrs):
self.show_with_level('startelementname:"%s"'%
(name,))
self.level+=1
defendElement(self,name):
self.level=1
self.show_with_level('endelementname:"%s"'%
(name,))
defcharacters(self,content):
content=content.strip()
ifcontent:
self.show_with_level('characters:"%s"'%
(content,))
parser=make_parser()
handler=TestHandler()
parser.setContentHandler(handler)
Page185
parser.parse(infilename)
defusage():
print__doc__
sys.exit(1)
defmain():
args=sys.argv[1:]
iflen(args)!=1:
usage()
infilename=args[0]
test(infilename)
main()
2. Theminidommodulecontainsaparse()functionthatenablesustoreadan
XMLdocumentandcreateaDOMtree:
"""ProcessanXMLdocumentwithminidom.
Showthedocumenttree.
Usage:
pythonminidom_walk.py[options]infilename
"""
importsys
fromxml.domimportminidom
defshow_tree(doc):
root=doc.documentElement
show_node(root,0)
defshow_node(node,level):
count=0
ifnode.nodeType==minidom.Node.ELEMENT_NODE:
show_level(level)
print'tag:%s'%(node.nodeName,)
forkeyinnode.attributes.keys():
attr=node.attributes.get(key)
show_level(level+1)
print'attributename:%svalue:"%s"'%
(attr.name,
attr.value,)
if(len(node.childNodes)==1and
node.childNodes[0].nodeType==
minidom.Node.TEXT_NODE):
show_level(level+1)
print'data:"%s"'%(node.childNodes[0].data,
)
forchildinnode.childNodes:
count+=1
show_node(child,level+1)
returncount
defshow_level(level):
forxinrange(level):
Page186
print'',
deftest():
args=sys.argv[1:]
iflen(args)!=1:
print__doc__
sys.exit(1)
docname=args[0]
doc=minidom.parse(docname)
show_tree(doc)
test()
3. ElementTreeenablesustoparseanXMLdocumentandcreateaDOMtree:
"""ProcessanXMLdocumentwithelementtree.
Usage:
pythonelementtree_walk.py[options]infilename
"""
importsys
fromxml.etreeimportElementTreeasetree
defshow_tree(doc):
root=doc.getroot()
show_node(root,0)
show_level(level)
print'tag:%s'%(node.tag,)
forkey,valueinnode.attrib.iteritems():
show_level(level+1)
print'attributename:%svalue:"%s"'%(key,
value,)
ifnode.text:
text=node.text.strip()
show_level(level+1)
print'text:"%s"'%(node.text,)
ifnode.tail:
tail=node.tail.strip()
show_level(level+1)
print'tail:"%s"'%(tail,)
forchildinnode.getchildren():
forxinrange(level):
print'',
deftest():
args=sys.argv[1:]
iflen(args)!=1:
print__doc__
Page187
sys.exit(1)
docname=args[0]
doc=etree.parse(docname)
show_tree(doc)
test()
1. lxmlenablesustoparseanXMLdocumentandcreateaDOMtree.Infact,
sincelxmlattemptstomimictheElementTreeAPI,ourcodeisverysimilarto
thatinthesolutiontotheElementTreeexercise:
Usage:
pythonlxml_walk.py[options]infilename
"""
#
#Imports:
importsys
fromlxmlimportetree
defshow_tree(doc):
root=doc.getroot()
show_node(root,0)
show_level(level)
show_level(level+1)
value,)
ifnode.text:
show_level(level+1)
ifnode.tail:
show_level(level+1)
forxinrange(level):
print'',
deftest():
args=sys.argv[1:]
iflen(args)!=1:
print__doc__
sys.exit(1)
docname=args[0]
Page188
doc=etree.parse(docname)
show_tree(doc)
test()
1. WecanmodifytheDOMtreeandwriteitouttoanewfile:
Modifythedocumenttreeandthenshowitagain.
WritethemodifiedXMLtreetoanewfile.
Usage:
pythonelementtree_walk.py[options]infilename
outfilename
Options:
Example:
pythonelementtree_walk.pymyxmldoc.xml
myotherxmldoc.xml
"""
importsys
importos
importgetopt
importtime
#UseElementTree.
fromxml.etreeimportElementTreeasetree
#OruncommenttouseLxml.
#fromlxmlimportetree
defshow_tree(doc):
root=doc.getroot()
show_node(root,0)
show_level(level)
show_level(level+1)
value,)
ifnode.text:
show_level(level+1)
ifnode.tail:
show_level(level+1)
Page189
forxinrange(level):
print'',
defmodify_tree(doc,tag,attrname,attrvalue):
root=doc.getroot()
modify_node(root,tag,attrname,attrvalue)
defmodify_node(node,tag,attrname,attrvalue):
ifnode.tag==tag:
node.attrib[attrname]=attrvalue
modify_node(child,tag,attrname,attrvalue)
deftest(indocname,outdocname):
doc=etree.parse(indocname)
show_tree(doc)
print''*50
date=time.ctime()
modify_tree(doc,'person','date',date)
show_tree(doc)
write_output=False
ifos.path.exists(outdocname):
response=raw_input('Outputfile(%s)exists.
Overwrite?(y/n):'%
outdocname)
ifresponse=='y':
write_output=True
else:
write_output=True
ifwrite_output:
doc.write(outdocname)
print'WrotemodifiedXMLtreeto%s'%outdocname
else:
print'Didnotwriteoutputfile.'
defusage():
print__doc__
sys.exit(1)
defmain():
args=sys.argv[1:]
try:
opts,args=getopt.getopt(args,'h',['help',
])
except:
usage()
foropt,valinopts:
usage()
iflen(args)!=2:
usage()
indocname=args[0]
outdocname=args[1]
test(indocname,outdocname)
main()
Notes:
Page190
TheabovesolutioncontainsanimportstatementforElementTreeand
anotherforlxml.Theoneforlxmliscommentedout,butyoucould
changethatifyouwishtouselxmlinsteadofElementTree.Thissolution
willworkthesamewaywitheitherElementTreeorlxml.
2. WhenweparseandXMLdocumentwithlxml,eachelement(node)hasan
xpath()method.
#test_xpath.py
fromlxmlimportetree
deftest():
doc=etree.parse('people.xml')
root=doc.getroot()
printroot.xpath("//name/text()")
printroot.xpath("//@id")
test()
And,whenweruntheabovecode,hereiswhatwesee:
$pythontest_xpath.py
['Alberta','Bernardo','Charlie']
['1','2','3']
FormoreonXPathsee:XMLPathLanguage(XPath)
http://www.w3.org/TR/xpath
3.9.2Relationaldatabaseaccess
YoucanfindinformationaboutdatabaseprogramminginPythonhere:Database
Programminghttp://wiki.python.org/moin/DatabaseProgramming/.
FordatabaseaccessweusethePythonDatabaseAPI.Youcanfindinformationabout
ithere:PythonDatabaseAPISpecificationv2.0
http://www.python.org/dev/peps/pep0249/.
TousethedatabaseAPIwedothefollowing:
1. Usethedatabaseinterfacemoduletocreateaconnectionobject.
2. Usetheconnectionobjecttocreateacursorobject.
3. UsethecursorobjecttoexecuteanSQLquery.
4. Retrieverowsfromthecursorobject,ifneeded.
5. Optionally,commitresultstothedatabase.
6. Closetheconnectionobject.
Ourexamplesusethegadflydatabase,whichiswritteninPython.Ifyouwanttouse
gadfly,youcanfindithere:http://gadfly.sourceforge.net/.gadflyisareasonable
choiceifyouwantaneasytousedatabaseonyourlocalmachine.
Anotherreasonablechoiceforalocaldatabaseissqlite3,whichisinthePython
standardlibrary.HereisadescriptivequotefromtheSQLiteWebsite:
"SQLiteisasoftwarelibrarythatimplementsaselfcontained,
serverless,zeroconfiguration,transactionalSQLdatabaseengine.
SQLiteisthemostwidelydeployedSQLdatabaseengineinthe
Page191
world.ThesourcecodeforSQLiteisinthepublicdomain."
Youcanlearnaboutithere:
sqlite3DBAPI2.0interfaceforSQLitedatabases
http://docs.python.org/library/sqlite3.html
SQLitehomepagehttp://www.sqlite.org/
Thepysqlitewebpagehttp://oss.itsystementwicklung.de/trac/pysqlite/
Ifyouwantorneedtouseanother,enterpriseclassdatabase,forexample
PostgreSQL,MySQL,Oracle,etc.,youwillneedaninterfacemoduleforyour
specificdatabase.Youcanfindinformationaboutdatabaseinterfacemoduleshere:
Databaseinterfaceshttp://wiki.python.org/moin/DatabaseInterfaces
Excercises:
1. Writeascriptthatretrievesalltherowsinatableandprintseachrow.
2. Writeascriptthatretrievesalltherowsinatable,thenusesthecursorasan
iteratortoprinteachrow.
3. Writeascriptthatusesthecursor'sdescriptionattributetoprintoutthename
andvalueofeachfieldineachrow.
4. Writeascriptthatperformsseveraloftheabovetasks,butusessqlite3instead
ofgadfly.
Solutions:
1. WecanexecuteaSQLqueryandthenretrievealltherowswithfetchall():
importgadfly
deftest():
connection=gadfly.connect("dbtest1","plantsdbdir")
cur=connection.cursor()
cur.execute('select*fromplantsdborderbyp_name')
rows=cur.fetchall()
forrowinrows:
print'2.row:',row
connection.close()
test()
2. Thecursoritselfisaniterator.Ititeratesovertherowsreturnedbyaquery.So,
weexecuteaSQLqueryandthenweusethecursorinafor:statement:
importgadfly
deftest():
connection=gadfly.connect("dbtest1","plantsdbdir")
cur=connection.cursor()
forrowincur:
printrow
connection.close()
test()
3. Thedescriptionattributeinthecursorisacontainerthathasanitem
describingeachfield:
Page192
importgadfly
deftest():
forfieldincur.description:
print'field:',field
rows=cur.fetchall()
forrowinrows:
foridx,fieldinenumerate(row):
content='%s:"%s"'%(cur.description[idx][0],
field,)
printcontent,
print
connection.close()
test()
Notes:
ThecommaattheendoftheprintstatementtellsPythonnottoprinta
newline.
Thecur.descriptionisasequencecontaininganitemforeachfield.After
thequery,wecanextractadescriptionofeachfield.
4. Thesolutionsusingsqlite3areverysimilartothoseusinggadfly:
"""
Performoperationsonsqlite3(plants)database.
Usage:
pythonpy_db_api.pycommand[arg1,...]
Commands:
createcreatenewdatabase.
showshowcontentsofdatabase.
addaddrowtodatabase.Requires3args(name,
descrip,rating).
deleteremoverowfromdatabase.Requires1arg
(name).
Examples:
pythontest1.pycreate
pythontest1.pyshow
pythontest1.pyaddcrenshaw"Themostsucculentmelon"
10
pythontest1.pydeletelemon
"""
importsys
importsqlite3
Values=[
('lemon','brightandyellow','7'),
('peach','succulent','9'),
('banana','smoothandcreamy','8'),
('nectarine','tangyandtasty','9'),
('orange','sweetandtangy','8'),
]
Field_defs=[
Page193
'p_namevarchar',
'p_descripvarchar',
#'p_ratinginteger',
'p_ratingvarchar',
]
defcreatedb():
connection=sqlite3.connect('sqlite3plantsdb')
cursor=connection.cursor()
q1="createtableplantsdb(%s)"%(',
'.join(Field_defs))
print'createq1:%s'%q1
cursor.execute(q1)
q1="createindexindex1onplantsdb(p_name)"
cursor.execute(q1)
q1="insertintoplantsdb(p_name,p_descrip,p_rating)
values('%s','%s',%s)"
forspecinValues:
q2=q1%spec
print'q2:"%s"'%q2
cursor.execute(q2)
connection.commit()
showdb1(cursor)
connection.close()
defshowdb():
connection,cursor=opendb()
showdb1(cursor)
connection.close()
defshowdb1(cursor):
cursor.execute("select*fromplantsdborderbyp_name")
hr()
description=cursor.description
printdescription
print'description:'
forrowdescriptionindescription:
print'%s'%(rowdescription,)
hr()
rows=cursor.fetchall()
printrows
print'rows:'
forrowinrows:
print'%s'%(row,)
hr()
print'content:'
forrowinrows:
descrip=row[1]
name=row[0]
rating='%s'%row[2]
print'%s%s%s'%(
name.ljust(12),descrip.ljust(30),
rating.rjust(4),)
defaddtodb(name,descrip,rating):
try:
Page194
rating=int(rating)
exceptValueError,exp:
print'Error:ratingmustbeinteger.'
return
cursor.execute("select*fromplantsdbwherep_name=
'%s'"%name)
iflen(rows)>0:
ql="updateplantsdbsetp_descrip='%s',
p_rating='%s'wherep_name='%s'"%(
descrip,rating,name,)
print'ql:',ql
cursor.execute(ql)
connection.commit()
print'Updated'
else:
cursor.execute("insertintoplantsdbvalues('%s',
'%s','%s')"%(
name,descrip,rating))
connection.commit()
print'Added'
showdb1(cursor)
connection.close()
defdeletefromdb(name):
cursor.execute("select*fromplantsdbwherep_name=
'%s'"%name)
iflen(rows)>0:
cursor.execute("deletefromplantsdbwhere
p_name='%s'"%name)
connection.commit()
print'Plant(%s)deleted.'%name
else:
print'Plant(%s)doesnotexist.'%name
showdb1(cursor)
connection.close()
defopendb():
connection=sqlite3.connect("sqlite3plantsdb")
cursor=connection.cursor()
returnconnection,cursor
defhr():
print''*60
defusage():
print__doc__
sys.exit(1)
defmain():
args=sys.argv[1:]
iflen(args)<1:
Page195
usage()
cmd=args[0]
ifcmd=='create':
iflen(args)!=1:
usage()
createdb()
elifcmd=='show':
iflen(args)!=1:
usage()
showdb()
elifcmd=='add':
iflen(args)<4:
usage()
name=args[1]
descrip=args[2]
rating=args[3]
addtodb(name,descrip,rating)
elifcmd=='delete':
iflen(args)<2:
usage()
name=args[1]
deletefromdb(name)
else:
usage()
main()
3.9.3CSVcommaseparatedvaluefiles
Exercises:
1. ReadaCSVfileandprintthefieldsincolumns.Hereisasamplefiletouseas
input:
#namedescriptionrating
Lemon,Brightyellowandtart,5
Eggplant,Purpleandshiny,6
Tangerine,Succulent,8
Solutions:
1. UsetheCSVmoduleinthePythonstandardlibrarytoreadaCSVfile:
"""
ReadaCSVfileandprintthecontentsincolumns.
"""
importcsv
infile=open(infilename)
reader=csv.reader(infile)
print'===============
======'
print'NameDescription
Rating'
print'===============
======'
forfieldsinreader:
Page196
iflen(fields)==3:
line='%s%s%s'%(fields[0].ljust(20),
fields[1].ljust(40),fields[2].ljust(4))
printline
infile.close()
defmain():
infilename='csv_report.csv'
test(infilename)
main()
And,whenrun,hereiswhatitdisplays:
===============
======
NameDescription
Rating
===============
======
LemonBrightyellowandtart
5
EggplantPurpleandshiny
6
TangerineSucculent
8
3.9.4YAMLandPyYAML
YAMLisastructuredtextdatarepresentationformat.Itusesindentationtoindicate
nesting.HereisadescriptionfromtheYAMLWebsite:
"YAML:YAMLAin'tMarkupLanguage
"WhatItIs:YAMLisahumanfriendlydataserializationstandard
forallprogramminglanguages."
YoucanlearnmoreaboutYAMLandPyYAMLhere:
TheOfficialYAMLWebSitehttp://yaml.org/
PyYAML.orgthehomeofvariousYAMLimplementationsforPython
http://pyyaml.org/
TheYAML1.2specificationhttp://yaml.org/spec/1.2/
Exercises:
1. ReadthefollowingsampleYAMLdocument.Printouttheinformationinit:
american:
BostonRedSox
DetroitTigers
NewYorkYankees
national:
NewYorkMets
ChicagoCubs
AtlantaBraves
2. LoadtheYAMLdatausedinthepreviousexercise,thenmakeamodification
Page197
(forexample,add"SanFranciscoGiants"totheNationalLeague),thendump
themodifieddatatoanewfile.
Solutions:
1. PrintingoutinformationfromYAMLisas"simple"asprintingoutaPython
datastructure.Inthissolution,weusetheprettyprinterfromthePython
standardlibrary:
importyaml
importpprint
deftest():
infile=open('test1.yaml')
data=yaml.load(infile)
infile.close()
pprint.pprint(data)
test()
Wecould,alternatively,readinandthen"load"fromastring:
importyaml
importpprint
deftest():
infile=open('test1.yaml')
data_str=infile.read()
infile.close()
data=yaml.load(data_str)
pprint.pprint(data)
test()
2. TheYAMLdump()functionenablesustodumpdatatoafile:
importyaml
importpprint
deftest():
infile=open('test1.yaml','r')
data=yaml.load(infile)
infile.close()
data['national'].append('SanFranciscoGiants')
outfile=open('test1_new.yaml','w')
yaml.dump(data,outfile)
outfile.close()
test()
Notes:
IfwewanttoproducethestandardYAML"block"styleratherthanthe
"flow"format,thenwecoulduse:
yaml.dump(data,outfile,default_flow_style=False)
3.9.5Json
HereisaquotefromWikipediaentryforJson:
Page198
"JSON(pronounced'Jason'),shortforJavaScriptObjectNotation,
isalightweightcomputerdatainterchangeformat.Itisatextbased,
humanreadableformatforrepresentingsimpledatastructuresand
associativearrays(calledobjects)."
TheJsontextrepresentationlooksverysimilartoPythonliteralrepresentationof
Pythonbuiltindatatypes(forexample,lists,dictionaries,numbers,andstrings).
LearnmoreaboutJsonandPythonsupportforJsonhere:
IntroducingJSONhttp://json.org/
JsonatWikipediahttp://en.wikipedia.org/wiki/Json
pythonjsonhttp://pypi.python.org/pypi/pythonjson
simplejsonhttp://pypi.python.org/pypi/simplejson
Excercises:
1. WriteaPythonscript,usingyourfavoritePythonJsonimplementation(for
examplepython-jsonorsimplejson),thatdumpsthefollowingdatastructure
toafile:
Data={
'rockandroll':
['Elis','TheBeatles','TheRollingStones',],
'country':
['WillieNelson','HankWilliams',]
}
2. WriteaPythonscriptthatreadsJsondatafromafileandloadsitintoPython
datastructures.
Solutions:
1. ThissolutionusessimplejsontostoreaPythondatastructureencodedasJson
inafile:
importsimplejsonasjson
Data={
'rockandroll':
['Elis','TheBeatles','TheRollingStones',],
'country':
['WillieNelson','HankWilliams',]
}
deftest():
fout=open('tmpdata.json','w')
content=json.dumps(Data)
fout.write(content)
fout.write('\n')
fout.close()
test()
2. Wecanreadthefileintoastring,thendecodeitfromJson:
importsimplejsonasjson
deftest():
fin=open('tmpdata.json','r')
Page199
content=fin.read()
fin.close()
data=json.loads(content)
printdata
test()
Notethatyoumaywantsomecontroloverindentation,characterencoding,etc.For
simplejson,youcanlearnaboutthathere:simplejsonJSONencoderanddecoder
http://simplejson.googlecode.com/svn/tags/simplejson2.0.1/docs/index.html.
Page200

Python Book 01

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Python Book 01

Uploaded by

Copyright:

Available Formats

APythonBook:BeginningPython,Advanced

Usefor line in inFile:toprocessonelineatatime.Youcandothisbecause

Recommendation:Usethisformsparingly.Usingfrom mod import *makes

mo.expand(r'value1: \1 value2: \2').

Thisexampleparsesexpressionsoftheformfunc(arg1, arg2, arg3):

You might also like