A Python Book 2014 PDF

APythonBook
APythonBook:BeginningPython,Advanced
Python,andPythonExercises
Author:
DaveKuhlman
Contact:
dkuhlman@davekuhlman.org
Address:
http://www.davekuhlman.org
Page1
APythonBook
Revision
1.3a
Date
December15,2013
Copyright
Copyright(c)2009DaveKuhlman.AllRightsReserved.Thisdocumentissubject
totheprovisionsoftheOpenSourceMITLicense
http://www.opensource.org/licenses/mitlicense.php.
Abstract
ThisdocumentisaselflearningdocumentforacourseinPythonprogramming.
Thiscoursecontains(1)apartforbeginners,(2)adiscussionofseveraladvanced
topicsthatareofinteresttoPythonprogrammers,and(3)aPythonworkbookwith
lotsofexercises.
Page2
APythonBook
Contents
1Part1BeginningPython...........................................................................................10
1.1IntroductionsEtc...................................................................................................10
1.1.1Resources.......................................................................................................11
1.1.2AgeneraldescriptionofPython....................................................................12
1.1.3InteractivePython..........................................................................................15
1.2Lexicalmatters......................................................................................................15
1.2.1Lines..............................................................................................................15
1.2.2Comments......................................................................................................16
1.2.3Namesandtokens..........................................................................................16
1.2.4Blocksandindentation..................................................................................16
1.2.5Docstrings.....................................................................................................17
1.2.6Programstructure..........................................................................................17
1.2.7Operators.......................................................................................................18
1.2.8Alsosee.........................................................................................................19
1.2.9Codeevaluation.............................................................................................19
1.3Statementsandinspectionpreliminaries...........................................................20
1.4Builtindatatypes.................................................................................................21
1.4.1Numerictypes................................................................................................21
1.4.2Tuplesandlists..............................................................................................21
1.4.3Strings............................................................................................................24
1.4.3.1Thenewstring.formatmethod...............................................................26
1.4.3.2Unicodestrings......................................................................................27
1.4.4Dictionaries....................................................................................................29
1.4.5Files...............................................................................................................32
1.4.6Otherbuiltintypes........................................................................................35
1.4.6.1TheNonevalue/type..............................................................................35
1.4.6.2Booleanvalues.......................................................................................36
1.4.6.3Setsandfrozensets.................................................................................36
1.5FunctionsandClassesAPreview......................................................................36
1.6Statements.............................................................................................................37
1.6.1Assignmentstatement....................................................................................37
1.6.2importstatement............................................................................................39
1.6.3printstatement...............................................................................................41
1.6.4if:elif:else:statement...................................................................................43
1.6.5for:statement.................................................................................................44
1.6.6while:statement.............................................................................................48
Page3
APythonBook
1.6.7continueandbreakstatements.......................................................................48
1.6.8try:except:statement.....................................................................................49
1.6.9raisestatement...............................................................................................51
1.6.10with:statement.............................................................................................52
1.6.10.1Writingacontextmanager...................................................................52
1.6.10.2Usingthewith:statement....................................................................53
1.6.11del................................................................................................................54
1.6.12casestatement..............................................................................................55
1.7Functions,Modules,Packages,andDebugging....................................................55
1.7.1Functions.......................................................................................................55
1.7.1.1Thedefstatement...................................................................................55
1.7.1.2Returningvalues....................................................................................55
1.7.1.3Parameters..............................................................................................56
1.7.1.4Arguments..............................................................................................56
1.7.1.5Localvariables.......................................................................................57
1.7.1.6Otherthingstoknowaboutfunctions....................................................57
1.7.1.7Globalvariablesandtheglobalstatement.............................................58
1.7.1.8Docstringsforfunctions.......................................................................60
1.7.1.9Decoratorsforfunctions........................................................................60
1.7.2lambda...........................................................................................................61
1.7.3Iteratorsandgenerators.................................................................................62
1.7.4Modules.........................................................................................................67
1.7.4.1Docstringsformodules.........................................................................68
1.7.5Packages........................................................................................................68
1.8Classes...................................................................................................................69
1.8.1Asimpleclass................................................................................................69
1.8.2Definingmethods..........................................................................................70
1.8.3Theconstructor..............................................................................................70
1.8.4Membervariables..........................................................................................70
1.8.5Callingmethods.............................................................................................71
1.8.6Addinginheritance........................................................................................71
1.8.7Classvariables...............................................................................................72
1.8.8Classmethodsandstaticmethods.................................................................72
1.8.9Properties.......................................................................................................74
1.8.10Interfaces.....................................................................................................75
1.8.11Newstyleclasses.........................................................................................75
1.8.12Docstringsforclasses.................................................................................77
1.8.13Privatemembers..........................................................................................77
1.9SpecialTasks.........................................................................................................77
1.9.1Debuggingtools.............................................................................................77
Page4
APythonBook
1.9.2Fileinputandoutput......................................................................................78
1.9.3Unittests........................................................................................................80
1.9.3.1Asimpleexample..................................................................................80
1.9.3.2Unittestsuites........................................................................................81
1.9.3.3Additionalunittestfeatures....................................................................83
1.9.3.4GuidanceonUnitTesting......................................................................85
1.9.4doctest............................................................................................................85
1.9.5ThePythondatabaseAPI..............................................................................87
1.9.6InstallingPythonpackages............................................................................88
1.10MorePythonFeaturesandExercises..................................................................89
2Part2AdvancedPython............................................................................................90
2.1IntroductionPython201(Slightly)AdvancedPythonTopics.......................90
2.2RegularExpressions..............................................................................................90
2.2.1Definingregularexpressions.........................................................................90
2.2.2Compilingregularexpressions......................................................................91
2.2.3Usingregularexpressions..............................................................................91
2.2.4Usingmatchobjectstoextractavalue..........................................................92
2.2.5Extractingmultipleitems..............................................................................93
2.2.6Replacingmultipleitems...............................................................................94
2.3IteratorObjects......................................................................................................96
2.3.1ExampleAgeneratorfunction....................................................................98
2.3.2ExampleAclasscontainingageneratormethod......................................100
2.3.3ExampleAniteratorclass.........................................................................102
2.3.4ExampleAniteratorclassthatusesyield.................................................104
2.3.5ExampleAlistcomprehension.................................................................105
2.3.6ExampleAgeneratorexpression..............................................................105
2.4UnitTests............................................................................................................106
2.4.1Definingunittests........................................................................................106
2.4.1.1Createatestclass.................................................................................106
2.5ExtendingandembeddingPython......................................................................109
2.5.1Introductionandconcepts............................................................................109
2.5.2Extensionmodules.......................................................................................110
2.5.3SWIG...........................................................................................................112
2.5.4Pyrex............................................................................................................115
2.5.5SWIGvs.Pyrex...........................................................................................120
2.5.6Cython.........................................................................................................120
2.5.7Extensiontypes............................................................................................122
2.5.8Extensionclasses.........................................................................................122
2.6Parsing.................................................................................................................122
2.6.1Specialpurposeparsers...............................................................................123
Page5
APythonBook
2.6.2Writingarecursivedescentparserbyhand.................................................124
2.6.3Creatingalexer/tokenizerwithPlex...........................................................131
2.6.4Asurveyofexistingtools............................................................................141
2.6.5CreatingaparserwithPLY.........................................................................141
2.6.6Creatingaparserwithpyparsing.................................................................148
2.6.6.1Parsingcommadelimitedlines............................................................148
2.6.6.2Parsingfunctors...................................................................................149
2.6.6.3Parsingnames,phonenumbers,etc.....................................................150
2.6.6.4Amorecomplexexample....................................................................151
2.7GUIApplications................................................................................................153
2.7.1Introduction.................................................................................................153
2.7.2PyGtk...........................................................................................................153
2.7.2.1Asimplemessagedialogbox..............................................................153
2.7.2.2Asimpletextinputdialogbox.............................................................156
2.7.2.3Afileselectiondialogbox...................................................................158
2.7.3EasyGUI......................................................................................................160
2.7.3.1AsimpleEasyGUIexample................................................................161
2.7.3.2AnEasyGUIfileopendialogexample................................................161
2.8GuidanceonPackagesandModules...................................................................161
2.8.1Introduction.................................................................................................161
2.8.2ImplementingPackages...............................................................................162
2.8.3UsingPackages............................................................................................162
2.8.4DistributingandInstallingPackages...........................................................162
2.9EndMatter...........................................................................................................164
2.9.1AcknowledgementsandThanks..................................................................164
2.9.2SeeAlso.......................................................................................................164
3Part3PythonWorkbook.........................................................................................165
3.1Introduction.........................................................................................................165
3.2LexicalStructures................................................................................................165
3.2.1Variablesandnames....................................................................................165
3.2.2Linestructure...............................................................................................167
3.2.3Indentationandprogramstructure...............................................................168
3.3ExecutionModel.................................................................................................169
3.4BuiltinDataTypes.............................................................................................170
3.4.1Numbers......................................................................................................170
3.4.1.1Literalrepresentationsofnumbers......................................................171
3.4.1.2Operatorsfornumbers.........................................................................173
3.4.1.3Methodsonnumbers............................................................................175
3.4.2Lists.............................................................................................................175
3.4.2.1Literalrepresentationoflists...............................................................176
Page6
APythonBook
3.4.2.2Operatorsonlists.................................................................................178
3.4.2.3Methodsonlists...................................................................................178
3.4.2.4Listcomprehensions............................................................................180
3.4.3Strings..........................................................................................................182
3.4.3.1Characters............................................................................................183
3.4.3.2Operatorsonstrings.............................................................................184
3.4.3.3Methodsonstrings...............................................................................185
3.4.3.4Rawstrings..........................................................................................187
3.4.3.5Unicodestrings....................................................................................188
3.4.4Dictionaries..................................................................................................190
3.4.4.1Literalrepresentationofdictionaries...................................................190
3.4.4.2Operatorsondictionaries.....................................................................191
3.4.4.3Methodsondictionaries.......................................................................192
3.4.5Files.............................................................................................................195
3.4.6Afewmiscellaneousdatatypes..................................................................197
3.4.6.1None.....................................................................................................197
3.4.6.2ThebooleansTrueandFalse...............................................................197
3.5Statements...........................................................................................................198
3.5.1Assignmentstatement..................................................................................198
3.5.2printstatement.............................................................................................200
3.5.3if:statementexercises..................................................................................201
3.5.4for:statementexercises...............................................................................202
3.5.5while:statementexercises...........................................................................205
3.5.6breakandcontinuestatements.....................................................................206
3.5.7Exceptionsandthetry:except:andraisestatements...................................207
3.6Functions.............................................................................................................210
3.6.1Optionalargumentsanddefaultvalues.......................................................211
3.6.2Passingfunctionsasarguments...................................................................213
3.6.3Extraargsandkeywordargs.......................................................................214
3.6.3.1Orderofarguments(positional,extra,andkeywordargs)..................216
3.6.4Functionsandducktypingandpolymorphism...........................................216
3.6.5Recursivefunctions.....................................................................................217
3.6.6Generatorsanditerators...............................................................................219
3.7Objectorientedprogrammingandclasses..........................................................223
3.7.1Theconstructor............................................................................................224
3.7.2InheritanceImplementingasubclass.......................................................225
3.7.3Classesandpolymorphism..........................................................................227
3.7.4Recursivecallstomethods..........................................................................228
3.7.5Classvariables,classmethods,andstaticmethods.....................................230
3.7.5.1Decoratorsforclassmethodandstaticmethod.....................................233
Page7
APythonBook
3.8AdditionalandAdvancedTopics........................................................................234
3.8.1Decoratorsandhowtoimplementthem......................................................234
3.8.1.1Decoratorswitharguments..................................................................235
3.8.1.2Stackeddecorators...............................................................................236
3.8.1.3Morehelpwithdecorators...................................................................238
3.8.2Iterables.......................................................................................................239
3.8.2.1AfewpreliminariesonIterables..........................................................239
3.8.2.2Morehelpwithiterables......................................................................240
3.9ApplicationsandRecipes....................................................................................240
3.9.1XMLSAX,minidom,ElementTree,Lxml..............................................241
3.9.2Relationaldatabaseaccess...........................................................................249
3.9.3CSVcommaseparatedvaluefiles...........................................................255
3.9.4YAMLandPyYAML..................................................................................256
3.9.5Json..............................................................................................................258
4Part4GeneratingPythonBindingsforXML.........................................................260
4.1Introduction.........................................................................................................260
4.2Generatingthecode.............................................................................................261
4.3UsingthegeneratedcodetoparseandexportanXMLdocument.....................263
4.4Somecommandlineoptionsyoumightwanttoknow.......................................263
4.5Thegraphicalfrontend.......................................................................................264
4.6Addingapplicationspecificbehavior.................................................................265
4.6.1Implementingcustomsubclasses................................................................265
4.6.2Usingthegenerated"API"fromyourapplication......................................266
4.6.3Acombinedapproach..................................................................................267
4.7Specialsituationsanduses..................................................................................269
4.7.1Generic,typeindependentprocessing.........................................................269
4.7.1.1Step1generatethebindings............................................................270
4.7.1.2Step2addapplicationspecificcode................................................270
4.7.1.3Step3writeatest/driverharness.....................................................274
4.7.1.4Step4runthetestapplication..........................................................276
4.8Somehints...........................................................................................................276
4.8.1ChildrendefinedwithmaxOccursgreaterthan1........................................276
4.8.2Childrendefinedwithsimplenumerictypes...............................................277
4.8.3Thetypeofanelement'scharactercontent..................................................277
4.8.4Constructorsandtheirdefaultvalues..........................................................277
Page8
APythonBook
Preface
ThisbookisacollectionofmaterialsthatI'veusedwhenconductingPythontrainingand
alsomaterialsfrommyWebsitethatareintendedforselfinstruction.
Youmaypreferamachinereadablecopyofthisbook.Youcanfinditinvariousformats
here:
HTMLhttp://www.davekuhlman.org/python_book_01.html
PDFhttp://www.davekuhlman.org/python_book_01.pdf
ODF/OpenOfficehttp://www.davekuhlman.org/python_book_01.odt
And,letmethankthestudentsinmyPythonclasses.Theirquestionsandsuggestions
wereagreathelpinthepreparationofthesematerials.
Page9
APythonBook
1Part1BeginningPython
1.1IntroductionsEtc
Introductions
Practicalmatters:restrooms,breakroom,lunchandbreaktimes,etc.
StartingthePythoninteractiveinterpreter.Also,IPythonandIdle.
Runningscripts
EditorsChooseaneditorwhichyoucanconfiguresothatitindentswith4spaces,not
tabcharacters.ForalistofeditorsforPython,see:
http://wiki.python.org/moin/PythonEditors.Afewpossibleeditors:
SciTEhttp://www.scintilla.org/SciTE.html.
MSWindowsonly(1)TextPadhttp://www.textpad.com;(2)UltraEdit
http://www.ultraedit.com/.
JedSeehttp://www.jedsoft.org/jed/.
EmacsSeehttp://www.gnu.org/software/emacs/and
http://www.xemacs.org/faq/xemacsfaq.html.
jEditRequiresabitofcustomizationforPythonSeehttp://jedit.org.
Vimhttp://www.vim.org/
Geanyhttp://www.geany.org/
Andmanymore.
Interactiveinterpreters:
python
ipython
Idle
IDEsAlsosee
http://en.wikipedia.org/wiki/List_of_integrated_development_environments_for_Python:
PyWinMSWindowsonly.Availableat:
http://sourceforge.net/projects/pywin32/.
WingIDESeehttp://wingware.com/wingide/.
Eclipsehttp://eclipse.org/.ThereisapluginthatsupportsPython.
KdevelopLinux/KDESeehttp://www.kdevelop.org/.
EricLinuxKDE?Seehttp://ericide.pythonprojects.org/index.html
EmacsandSciTEwillevaluateaPythonbufferwithintheeditor.
Page10
APythonBook
1.1.1Resources
Whereelsetogethelp:
Pythonhomepagehttp://www.python.org
Pythonstandarddocumentationhttp://www.python.org/doc/.
Youwillalsofindlinkstotutorialsthere.
FAQshttp://www.python.org/doc/faq/.
ThePythonWikihttp://wiki.python.org/
ThePythonPackageIndexLotsofPythonpackages
https://pypi.python.org/pypi
Specialinterestgroups(SIGs)http://www.python.org/sigs/
Otherpythonrelatedmailinglistsandlistsforspecificapplications(forexample,
Zope,Twisted,etc).Try:http://dir.gmane.org/search.php?match=python.
http://sourceforge.netLotsofprojects.Searchfor"python".
USENETcomp.lang.python.CanalsobeaccessedthroughGmane:
http://dir.gmane.org/gmane.comp.python.general.
ThePythontutoremaillisthttp://mail.python.org/mailman/listinfo/tutor
Localdocumentation:
OnMSWindows,thePythondocumentationisinstalledwiththestandard
installation.
InstallthestandardPythondocumentationonyourmachinefrom
http://www.python.org/doc/.
pydoc.Example,onthecommandline,type:pydocre.
Importamodule,thenviewits.__doc__attribute.
Attheinteractiveprompt,usehelp(obj).Youmightneedtoimportitfirst.
Example:
>>>importurllib
>>>help(urllib)
InIPython,thequestionmarkoperatorgiveshelp.Example:
In[13]:open?
Type:builtin_function_or_method
BaseClass:<type'builtin_function_or_method'>
StringForm:<builtinfunctionopen>
Namespace:Pythonbuiltin
Docstring:
open(name[,mode[,buffering]])>fileobject
Openafileusingthefile()type,returnsafile
object.
ConstructorDocstring:
x.__init__(...)initializesx;see
x.__class__.__doc__forsignature
Page11
APythonBook
Callable:Yes
Calldef:Callingdefinitionnotavailable.Call
docstring:
x.__call__(...)<==>x(...)
1.1.2AgeneraldescriptionofPython
Pythonisahighlevelgeneralpurposeprogramminglanguage:
Becausecodeisautomaticallycompiledtobytecodeandexecuted,Pythonis
suitableforuseasascriptinglanguage,Webapplicationimplementation
language,etc.
BecausePythoncanbeextendedinCandC++,Pythoncanprovidethespeed
neededforevencomputeintensivetasks.
Becauseofitsstrongstructuringconstructs(nestedcodeblocks,functions,
classes,modules,andpackages)anditsconsistentuseofobjectsand
objectorientedprogramming,Pythonenablesustowriteclear,logical
applicationsforsmallandlargetasks.
ImportantfeaturesofPython:
Builtinhighleveldatatypes:strings,lists,dictionaries,etc.
Theusualcontrolstructures:if,ifelse,ifelifelse,while,plusapowerful
collectioniterator(for).
Multiplelevelsoforganizationalstructure:functions,classes,modules,and
packages.Theseassistinorganizingcode.Anexcellentandlargeexampleisthe
Pythonstandardlibrary.
CompileontheflytobytecodeSourcecodeiscompiledtobytecodewithouta
separatecompilestep.Sourcecodemodulescanalsobe"precompiled"tobyte
codefiles.
ObjectorientedPythonprovidesaconsistentwaytouseobjects:everythingis
anobject.And,inPythonitiseasytoimplementnewobjecttypes(calledclasses
inobjectorientedprogramming).
ExtensionsinCandC++Extensionmodulesandextensiontypescanbewritten
byhand.Therearealsotoolsthathelpwiththis,forexample,SWIG,sip,Pyrex.
JythonisaversionofPythonthat"playswellwith"Java.See:TheJythonProject
http://www.jython.org/Project/.
Somethingsyouwillneedtoknow:
Pythonusesindentationtoshowblockstructure.Indentoneleveltoshowthe
beginningofablock.Outdentoneleveltoshowtheendofablock.Asan
example,thefollowingCstylecode:
if(x)
{
Page12
APythonBook
if(y)
{
f1()
}
f2()
}
inPythonwouldbe:
ifx:
ify:
f1()
f2()
And,theconventionistousefourspaces(andnohardtabs)foreachlevelofindentation.
Actually,it'smorethanaconvention;it'spracticallyarequirement.Followingthat
"convention"willmakeitsomucheasiertomergeyourPythoncodewithcodefrom
othersources.
AnoverviewofPython:
AscriptinglanguagePythonissuitable(1)forembedding,(2)forwritingsmall
unstructuredscripts,(3)for"quickanddirty"programs.
Notascriptinglanguage(1)Pythonscales.(2)Pythonencouragesustowrite
codethatisclearandwellstructured.
Interpreted,butalsocompiledtobytecode.Modulesareautomaticallycompiled
(to.pyc)whenimported,butmayalsobeexplicitlycompiled.
Providesaninteractivecommandlineandinterpretershell.Infact,thereare
several.
DynamicForexample:
Typesareboundtovalues,nottovariables.
Functionandmethodlookupisdoneatruntime.
Valuesareinspectable.
Thereisaninteractiveinterpreter,morethanone,infact.
Youcanlistthemethodssupportedbyanygivenobject.
Stronglytypedatruntime,notcompiletime.Objects(values)haveatype,but
variablesdonot.
ReasonablyhighlevelHighlevelbuiltindatatypes;highlevelcontrol
structures(forwalkinglistsanditerators,forexample).
ObjectorientedAlmosteverythingisanobject.Simpleobjectdefinition.Data
hidingbyagreement.Multipleinheritance.Interfacesbyconvention.
Polymorphism.
HighlystructuredStatements,functions,classes,modules,andpackagesenable
ustowritelarge,wellstructuredapplications.Whystructure?Readability,
locateability,modifiability.
Explicitness
Page13
APythonBook
Firstclassobjects:
Definition:Can(1)passtofunction;(2)returnfromfunction;(3)stuffintoa
datastructure.
Operatorscanbeappliedtovalues(notvariables).Example:f(x)[3]
Indentedblockstructure"Pythonispseudocodethatruns."
EmbeddingandextendingPythonPythonprovidesawelldocumentedand
supportedway(1)toembedthePythoninterpreterinC/C++applicationsand(2)
toextendPythonwithmodulesandobjectsimplementedinC/C++.
Insomecases,SWIGcangeneratewrappersforexistingC/C++code
automatically.Seehttp://www.swig.org/
CythonenablesustogenerateCcodefromPythonandto"easily"create
wrappersforC/C++functions.See
http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/
ToembedandextendPythonwithJava,thereisJython.See
http://www.jython.org/
Automaticgarbagecollection.(But,thereisagcmoduletoallowexplicitcontrol
ofgarbagecollection.)
Comparisonwithotherlanguages:compiledlanguages(e.g.C/C++);Java;Perl,
Tcl,andRuby.Pythonexcellsat:developmentspeed,executionspeed,clarityand
maintainability.
VarietiesofPython:
CPythonStandardPython2.ximplementedinC.
JythonPythonfortheJavaenvironmenthttp://www.jython.org/
PyPyPythonwithaJITcompilerandstacklessmodehttp://pypy.org/
StacklessPythonwithenhancedthreadsupportandmicrothreadsetc.
http://www.stackless.com/
IronPythonPythonfor.NETandtheCLRhttp://ironpython.net/
Python3Thenew,newPython.Thisisintendedasareplacementfor
Python2.x.http://www.python.org/doc/.Afewdifferences(fromPython
2.x):
Theprintstatementchangedtotheprintfunction.
Stringsareunicodebydefault.
Classesareall"newstyle"classes.
Changestosyntaxforcatchingexceptions.
Changestointegersnolonginteger;integerdivisionwithautomatic
converttofloat.
Morepervasiveuseofiterables(ratherthancollections).
Etc.
ForamoreinformationaboutdifferencesbetweenPython2.xandPython3.x,
seethedescriptionofthevariousfixesthatcanbeappliedwiththe2to3tool:
Page14
APythonBook
http://docs.python.org/3/library/2to3.html#fixers
Themigrationtool,2to3,easestheconversionof2.xcodeto3.x.
AlsoseeTheZenofPythonhttp://www.python.org/peps/pep0020.html.Or,at
thePythoninteractiveprompt,type:
>>>importthis
1.1.3InteractivePython
IfyouexecutePythonfromthecommandlinewithnoscript(noarguments),Python
givesyouaninteractiveprompt.ThisisanexcellentfacilityforlearningPythonandfor
tryingsmallsnippetsofcode.Manyoftheexamplesthatfollowweredevelopedusing
thePythoninteractiveprompt.
StartthePythoninteractiveinterpreterbytypingpythonwithnoargumentsatthe
commandline.Forexample:
$python
Python2.6.1(r261:67515,Jan112009,15:19:23)
[GCC4.3.2]onlinux2
Type"help","copyright","credits"or"license"formore
information.
>>>print'hello'
hello
>>>
YoumayalsowanttoconsiderusingIDLE.IDLEisagraphicalintegrateddevelopment
environmentforPython;itcontainsaPythonshell.ItislikelythatIdlewasinstalledfor
youwhenyouinstalledPython.YouwillfindascripttostartupIDLEinthe
Tools/scriptsdirectoryofyourPythondistribution.IDLErequiresTkinter.
Inaddition,therearetoolsthatwillgiveyouamorepowerfulandfancyPython
interactiveinterpreter.OneexampleisIPython,whichisavailableat
http://ipython.scipy.org/.
1.2Lexicalmatters
1.2.1Lines
Pythondoeswhatyouwantittodomostofthetimesothatyouonlyhavetoadd
extracharacterssomeofthetime.
Statementseparatorisasemicolon,butisonlyneededwhenthereismorethan
onestatementonaline.And,writingmorethanonestatementonthesamelineis
consideredbadform.
ContinuationlinesAbackslashaslastcharacterofthelinemakesthe
Page15
APythonBook
followinglineacontinuationofthecurrentline.But,notethatanopening
"context"(parenthesis,squarebracket,orcurlybracket)makesthebackslash
unnecessary.
1.2.2Comments
Everythingafter"#"onalineisignored.Noblockcomments,butdocstringsarea
commentinquotesatthebeginningofamodule,class,methodorfunction.Also,editors
withsupportforPythonoftenprovidetheabilitytocommentoutselectedblocksofcode,
usuallywith"##".
1.2.3Namesandtokens
Allowedcharacters:azAZ09underscore,andmustbeginwithaletteror
underscore.
Namesandidentifiersarecasesensitive.
Identifierscanbeofunlimitedlength.
Specialnames,customizing,etc.Usuallybeginandendindoubleunderscores.
SpecialnameclassesSingleanddoubleunderscores.
SingleleadingsingleunderscoreSuggestsa"private"methodorvariable
name.Notimportedby"frommoduleimport*".
SingletrailingunderscoreUseittoavoidconflictswithPythonkeywords.
DoubleleadingunderscoresUsedinaclassdefinitiontocausename
mangling(weakhiding).But,notoftenused.
NamingconventionsNotrigid,but:
Modulesandpackagesalllowercase.
GlobalsandconstantsUppercase.
ClassesBumpycapswithinitialupper.
MethodsandfunctionsAlllowercasewithwordsseparatedbyunderscores.
LocalvariablesLowercase(withunderscorebetweenwords)orbumpy
capswithinitialloweroryourchoice.
GoodadviceFollowtheconventionsusedinthecodeonwhichyouare
working.
Names/variablesinPythondonothaveatype.Valueshavetypes.
1.2.4Blocksandindentation
Pythonrepresentsblockstructureandnestedblockstructurewithindentation,notwith
beginandendbrackets.
TheemptyblockUsethepassnoopstatement.
Benefitsoftheuseofindentationtoindicatestructure:
Page16
APythonBook
Reducestheneedforacodingstandard.Onlyneedtospecifythatindentationis4
spacesandnohardtabs.
Reducesinconsistency.Codefromdifferentsourcesfollowthesameindentation
style.Ithasto.
Reduceswork.Onlyneedtogettheindentationcorrect,notbothindentationand
brackets.
Reducesclutter.Eliminatesallthecurlybrackets.
Ifitlookscorrect,itiscorrect.Indentationcannotfoolthereader.
EditorconsiderationsThestandardis4spaces(nohardtabs)foreachindentationlevel.
Youwillneedatexteditorthathelpsyourespectthat.
1.2.5Docstrings
Docstringsarelikecomments,buttheyarecarriedwithexecutingcode.Docstringscan
beviewedwithseveraltools,e.g.help(),obj.__doc__,and,inIPython,aquestion
mark(?)afteranamewillproducehelp.
Adocstringiswrittenasaquotedstringthatisatthetopofamoduleorthefirstlines
aftertheheaderlineofafunctionorclass.
Wecanusetriplequotingtocreatedocstringsthatspanmultiplelines.
Therearealsotoolsthatextractandformatdocstrings,forexample:
pydocDocumentationgeneratorandonlinehelpsystem
http://docs.python.org/lib/modulepydoc.html.
epydocEpydoc:AutomaticAPIDocumentationGenerationforPython
http://epydoc.sourceforge.net/index.html
SphinxCanalsoextractdocumentationfromPythondocstrings.See
http://sphinxdoc.org/index.html.
Seethefollowingforsuggestionsandmoreinformationondocstrings:Docstring
conventionshttp://www.python.org/dev/peps/pep0257/.
1.2.6Programstructure
Executiondef,class,etcareexecutablestatementsthataddsomethingtothe
currentnamespace.Modulescanbebothexecutableandimportable.
Statements,datastructures,functions,classes,modules,packages.
Functions
Classes
Modulescorrespondtofileswitha"*.py"extension.Packagescorrespondtoa
directory(orfolder)inthefilesystem;apackagecontainsafilenamed
"__init__.py".Bothmodulesandpackagescanbeimported(seesectionimport
Page17
APythonBook
statement).
PackagesAdirectorycontainingafilenamed"__init__.py".Canprovide
additionalinitializationwhenthepackageoramoduleinitisloaded(imported).
1.2.7Operators
See:http://docs.python.org/ref/operators.html.Pythondefinesthefollowing
operators:
+***///%
<<>>&|^~
<><=>===!=<>
Thecomparisonoperators<>and!=arealternatespellingsofthesameoperator.
!=isthepreferredspelling;<>isobsolescent.
Logicaloperators:
andorisnotin
Therearealso(1)thedotoperator,(2)thesubscriptoperator[],andthe
function/methodcalloperator().
Forinformationontheprecedencesofoperators,seethetableat
http://docs.python.org/2/reference/expressions.html#operatorprecedence,which
isreproducedbelow.
Forinformationonwhatthedifferentoperatorsdo,thesectioninthe"Python
LanguageReference"titled"Specialmethodnames"maybeofhelp:
http://docs.python.org/2/reference/datamodel.html#specialmethodnames
ThefollowingtablesummarizestheoperatorprecedencesinPython,fromlowest
precedence(leastbinding)tohighestprecedence(mostbinding).Operatorsonthe
samelinehavethesameprecedence.Unlessthesyntaxisexplicitlygiven,
operatorsarebinary.Operatorsonthesamelinegrouplefttoright(exceptfor
comparisons,includingtests,whichallhavethesameprecedenceandchainfrom
lefttorightseesection5.9andexponentiation,whichgroupsfromrightto
left):
OperatorDescription
==========================================
lambdaLambdaexpression
orBooleanOR
andBooleanAND
notxBooleanNOT
in,notinMembershiptests
is,isnotIdentitytests
<,<=,>,>=,<>,!=,==Comparisons
|BitwiseOR
^BitwiseXOR
&BitwiseAND
<<,>>Shifts
Page18
APythonBook
+,Additionandsubtraction
*,/,%Multiplication,division,
remainder
+x,xPositive,negative
~xBitwisenot
**Exponentiation
x.attributeAttributereference
x[index]Subscription
x[index:index]Slicing
f(arguments...)Functioncall
(expressions...)Bindingortupledisplay
[expressions...]Listdisplay
{key:datum...}Dictionarydisplay
èxpressions...`Stringconversion
Notethatmostoperatorsresultincallstomethodswithspecialnames,for
example__add__,__sub__,__mul__,etc.SeeSpecialmethodnames
http://docs.python.org/2/reference/datamodel.html#specialmethodnames
Later,wewillseehowtheseoperatorscanbeemulatedinclassesthatyoudefine
yourself,throughtheuseofthesespecialnames.
1.2.8Alsosee
FormoreonlexicalmattersandPythonstyles,see:
CodeLikeaPythonista:IdiomaticPython
http://python.net/~goodger/projects/pycon/2007/idiomatic/handout.html.
StyleGuideforPythonCodehttp://www.python.org/dev/peps/pep0008/
Theflake8stylecheckingprogram.Seehttps://pypi.python.org/pypi/flake8.Also
seethepylintcodechecker:https://pypi.python.org/pypi/pylint.
1.2.9Codeevaluation
UnderstandingthePythonexecutionmodelHowPythonevaluatesandexecutesyour
code.
Evaluatingexpressions.
Creatingnames/variablesBindingThefollowingallcreatenames(variables)and
bindvalues(objects)tothem:(1)assignment,(2)functiondefinition,(3)classdefinition,
(4)functionandmethodcall,(5)importingamodule,...
FirstclassobjectsAlmostallobjectsinPythonarefirstclass.Definition:Anobjectis
firstclassif:(1)wecanputitinastructuredobject;(2)wecanpassittoafunction;(3)
wecanreturnitfromafunction.
ReferencesObjects(orreferencestothem)canbeshared.Whatdoesthismean?
Theobject(s)satisfytheidentitytestoperatoris.
Page19
APythonBook
Thebuiltinfunctionid()returnsthesamevalue.
Theconsequencesformutableobjectsaredifferentfromthoseforimmutable
objects.
Changing(updating)amutableobjectreferencedthroughonevariableor
containeralsochangesthatobjectreferencedthroughothervariablesor
containers,becauseitisthesameobject.
del()Thebuiltinfunctiondel()removesareference,not(necessarily)the
objectitself.
1.3Statementsandinspectionpreliminaries
printExample:
printobj
print"one","two",'three'
for:Example:
stuff=['aa','bb','cc']
foriteminstuff:
printitem
LearnwhatthetypeofanobjectisExample:
type(obj)
Learnwhatattributesanobjecthasandwhatit'scapabilitiesareExample:
dir(obj)
value="amessage"
dir(value)
GethelponaclassoranobjectExample:
help(str)
help("")
value="abc"
help(value)
help(value.upper)
InIPython(butnotstandardPython),youcanalsogethelpattheinteractivepromptby
typing"?"and"??"afteranobject.Example:
In[48]:a=''
In[49]:a.upper?
Type:builtin_function_or_method
StringForm:<builtinmethodupperofstrobjectat0x7f1c426e0508>
Docstring:
S.upper()>string
Page20
APythonBook
ReturnacopyofthestringSconvertedtouppercase.
1.4Builtindatatypes
Forinformationonbuiltindatatypes,seesectionBuiltinTypes
http://docs.python.org/lib/types.htmlinthePythonstandarddocumentation.
1.4.1Numerictypes
Thenumerictypesare:
PlainintegersSameprecisionasaClong,usuallya32bitbinarynumber.
LongintegersDefinewith100L.But,plainintegersareautomatically
promotedwhenneeded.
FloatsImplementedasaCdouble.Precisiondependsonyourmachine.See
sys.float_info.
ComplexnumbersDefinewith,forexample,3jorcomplex(3.0,2.0).
See2.3.4NumericTypesint,float,long,complex
http://docs.python.org/lib/typesnumeric.html.
Pythondoesmixedarithmetic.
Integerdivisiontruncates.ThisischangedinPython3.Usefloat(n)toforcecoercion
toafloat.Example:
In[8]:a=4
In[9]:b=5
In[10]:a/b
Out[10]:0#possiblywrong?
In[11]:float(a)/b
Out[11]:0.8
Applyingthefunctioncalloperator(parentheses)toatypeorclasscreatesaninstanceof
thattypeorclass.
ScientificandheavilynumericprogrammingHighlevelPythonisnotveryefficientfor
numericalprogramming.But,therearelibrariesthathelpNumpyandSciPySee:
SciPy:ScientificToolsforPythonhttp://scipy.org/
1.4.2Tuplesandlists
ListAlistisadynamicarray/sequence.Itisorderedandindexable.Alistismutable.
Listconstructors:[],list().
range()andxrange():
Page21
APythonBook
range(n)createsalistofnintegers.Optionalargumentsarethestartinginteger
andastride.
xrangeislikerange,exceptthatitcreatesaniteratorthatproducestheitems
inthelistofintegersinsteadofthelistitself.
TuplesAtupleisasequence.Atupleisimmutable.
Tupleconstructors:(),butreallyacomma;alsotuple().
Tuplesarelikelists,butarenotmutable.
Pythonlistsare(1)heterogeneous(2)indexable,and(3)dynamic.Forexample,wecan
addtoalistandmakeitlonger.
Notesonsequenceconstructors:
Toconstructatuplewithasingleelement,use(x,);atuplewithasingle
elementrequiresacomma.
Youcanspreadelementsacrossmultiplelines(andnoneedforbackslash
continuationcharacter"\").
Acommacanfollowthelastelement.
Thelengthofatupleorlist(orothercontainer):len(mylist).
Operatorsforlists:
Try:list1+list2,list1*n,list1+=list2,etc.
Comparisonoperators:<,==,>=,etc.
Testformembershipwiththeinoperator.Example:
In[77]:a=[11,22,33]
In[78]:a
Out[78]:[11,22,33]
In[79]:22ina
Out[79]:True
In[80]:44ina
Out[80]:False
Subscription:
Indexingintoasequence
NegativeindexesEffectively,lengthofsequenceplus(minus)index.
SlicingExample:data[2:5].Defaultvalues:beginningandendoflist.
SlicingwithstridesExample:data[::2].
OperationsontuplesNooperationsthatchangethetuple,sincetuplesareimmutable.
Wecandoiterationandsubscription.Wecando"contains"(theinoperator)andgetthe
length(thelen()operator).Wecanusecertainbooleanoperators.
OperationsonlistsOperationssimilartotuplesplus:
Appendmylist.append(newitem).
Page22
APythonBook
Insertmylist.insert(index,newitem).Noteonefficiency:The
insertmethodisnotasfastastheappendmethod.Ifyoufindthatyouneed
todoalargenumberofmylist.insert(0,obj)(thatis,insertingatthe
beginningofthelist)considerusingadequeinstead.See:
http://docs.python.org/2/library/collections.html#collections.deque.Or,use
appendandreverse.
Extendmylist.extend(anotherlist).Alsocanuse+and+=.
Removemylist.remove(item)andmylist.pop().Notethat
append()togetherwithpop()implementsastack.
Deletedelmylist[index].
PopGetlast(rightmost)itemandremovefromlistmylist.pop().
Listoperators+,*,etc.
Formoreoperationsandoperatorsonsequences,see:
http://docs.python.org/2/library/stdtypes.html#sequencetypesstrunicodelisttuplebyte
arraybufferxrange.
Exercises:
Createanemptylist.Append4stringstothelist.Thenpoponeitemofftheend
ofthelist.Solution:
In[25]:a=[]
In[26]:a.append('aaa')
In[27]:a.append('bbb')
In[28]:a.append('ccc')
In[29]:a.append('ddd')
In[30]:printa
['aaa','bbb','ccc','ddd']
In[31]:a.pop()
Out[31]:'ddd'
Usetheforstatementtoprinttheitemsinthelist.Solution:
In[32]:foritemina:
....:printitem
....:
aaa
bbb
ccc
Usethestringjoinoperationtoconcatenatetheitemsinthelist.Solution:
In[33]:'||'.join(a)
Out[33]:'aaa||bbb||ccc'
Uselistscontainingthree(3)elementstocreateandshowatree:
In[37]:b=['bb',None,None]
In[38]:c=['cc',None,None]
In[39]:root=['aa',b,c]
Page23
APythonBook
In[40]:
In[40]:
In[40]:defshow_tree(t):
....:ifnott:
....:return
....:printt[0]
....:show_tree(t[1])
....:show_tree(t[2])
....:
....:
In[41]:show_tree(root)
aa
bb
cc
Notethatwewilllearnabetterwaytorepresenttreestructureswhenwecover
implementingclassesinPython.
1.4.3Strings
Stringsaresequences.Theyareimmutable.Theyareindexable.Theyareiterable.
Foroperationsonstrings,seehttp://docs.python.org/lib/stringmethods.htmloruse:
>>>help(str)
Or:
>>>dir("abc")
Stringoperations(methods).
Stringoperators,e.g.+,<,<=,==,etc..
Constructors/literals:
Quotes:singleanddouble.Escapingquotesandotherspecialcharacterswitha
backslash.
TriplequotingUsetriplesinglequotesordoublequotestodefinemultiline
strings.
str()Theconstructorandthenameofthetype/class.
'aSeparator'.join(aList)
Manymore.
Escapecharactersinstrings\t,\n,\\,etc.
StringformattingSee:
http://docs.python.org/2/library/stdtypes.html#stringformattingoperations
Examples:
In[18]:name='dave'
Page24
APythonBook
In[19]:size=25
In[20]:factor=3.45
In[21]:print'Name:%sSize:%dFactor:%3.4f'%(name,size,
factor,)
Name:daveSize:25Factor:3.4500
In[25]:print'Name:%sSize:%dFactor:%08.4f'%(name,size,
factor,)
Name:daveSize:25Factor:003.4500
Iftherighthandargumenttotheformattingoperatorisadictionary,thenyoucan
(actually,must)usethenamesofkeysinthedictionaryinyourformatstrings.Examples:
In[115]:values={'vegetable':'chard','fruit':'nectarine'}
In[116]:'Ilove%(vegetable)sandIlove%(fruit)s.'%values
Out[116]:'IlovechardandIlovenectarine.'
Alsoconsiderusingtherightjustifyandleftjustifyoperations.Examples:
mystring.rjust(20),mystring.ljust(20,':').
InPython3,thestr.formatmethodispreferredtothestringformattingoperator.
ThismethodisalsoavailableinPython2.7.Ithasbenefitsandadvantagesoverthestring
formattingoperator.Youcanstartlearningaboutithere:
http://docs.python.org/2/library/stdtypes.html#stringmethods
Exercises:
Usealiteraltocreateastringcontaining(1)asinglequote,(2)adoublequote,(3)
bothasingleanddoublequote.Solutions:
"Some'quoted'text."
'Some"quoted"text.'
'Some"quoted"\'extra\'text.'
Writeastringliteralthatspansmultiplelines.Solution:
"""Thisstring
spansseverallines
becauseitisalittlelong.
"""
Usethestringjoinoperationtocreateastringthatcontainsacolonasa
separator.Solution:
>>>content=[]
>>>content.append('finch')
>>>content.append('sparrow')
>>>content.append('thrush')
>>>content.append('jay')
>>>contentstr=':'.join(content)
>>>printcontentstr
finch:sparrow:thrush:jay
Usestringformattingtoproduceastringcontainingyourlastandfirstnames,
Page25
APythonBook
separatedbyacomma.Solution:
>>>first='Dave'
>>>last='Kuhlman'
>>>full='%s,%s'%(last,first,)
>>>printfull
Kuhlman,Dave
IncrementallybuildinguplargestringsfromlotsofsmallstringstheoldwaySince
stringsinPythonareimmutable,appendingtoastringrequiresareallocation.So,itis
fastertoappendtoalist,thenusejoin.Example:
In[25]:strlist=[]
In[26]:strlist.append('Line#1')
In[29]:str='\n'.join(strlist)
In[30]:printstr
Line#1
Line#2
Line#3
IncrementallybuildinguplargestringsfromlotsofsmallstringsthenewwayThe
+=operationonstringshasbeenoptimized.So,whenyoudothisstr1+=str2,
evenmanytimes,itisefficient.
Thetranslatemethodenablesustomapthecharactersinastring,replacingthosein
onetablebythoseinanother.And,themaketransfunctioninthestringmodule,
makesiteasytocreatethemappingtable:
importstring
deftest():
a='axbycz'
t=string.maketrans('abc','123')
printa
printa.translate(t)
test()
1.4.3.1Thenewstring.formatmethod
Thenewwaytodostringformatting(whichisstandardinPython3andperhaps
preferredfornewcodeinPython2)istousethestring.formatmethod.Seehere:
http://docs.python.org/2/library/stdtypes.html#str.format
http://docs.python.org/2/library/string.html#formatstringsyntax
http://docs.python.org/2/library/string.html#formatspecificationminilanguage
Someexamples:
Page26
APythonBook
In[1]:'aaa{1}bbb{0}ccc{1}ddd'.format('xx','yy',)
Out[1]:'aaayybbbxxcccyyddd'
In[2]:'number:{0:05d}ok'.format(25)
Out[2]:'number:00025ok'
In[4]:'n1:{num1}n2:{num2}'.format(num2=25,num1=100)
Out[4]:'n1:100n2:25'
In[5]:'n1:{num1}n2:{num2}again:{num1}'.format(num2=25,
num1=100)
Out[5]:'n1:100n2:25again:100'
In[6]:'number:{:05d}ok'.format(25)
Out[6]:'number:00025ok'
In[7]:values={'name':'dave','hobby':'birding'}
In[8]:'user:{name}activity:{hobby}'.format(**values)
Out[8]:'user:daveactivity:birding'
1.4.3.2Unicodestrings
Representingunicode:
In[96]:a=u'abcd'
In[97]:a
Out[97]:u'abcd'
In[98]:b=unicode('efgh')
In[99]:b
Out[99]:u'efgh'
Converttounicode:a_string.decode(encoding).Examples:
In[102]:'abcd'.decode('utf8')
Out[102]:u'abcd'
In[103]:
In[104]:'abcd'.decode(sys.getdefaultencoding())
Out[104]:u'abcd'
Convertoutofunicode:a_unicode_string.encode(encoding).Examples:
In[107]:a=u'abcd'
In[108]:a.encode('utf8')
Out[108]:'abcd'
In[109]:a.encode(sys.getdefaultencoding())
Out[109]:'abcd'
In[110]:b=u'Sel\xe7uk'
In[111]:printb.encode('utf8')
Seluk
TestforunicodetypeExample:
In[122]:importtypes
In[123]:a=u'abcd'
In[124]:type(a)istypes.UnicodeType
Out[124]:True
In[125]:
Page27
APythonBook
In[126]:type(a)istype(u'')
Out[126]:True
Orbetter:
In[127]:isinstance(a,unicode)
Out[127]:True
Anexamplewithacharacter"c"withahachek:
In[135]:name='IvanKrsti\xc4\x87'
In[136]:name.decode('utf8')
Out[136]:u'IvanKrsti\u0107'
In[137]:
In[138]:len(name)
Out[138]:12
In[139]:len(name.decode('utf8'))
Out[139]:11
Youcanalsocreateaunicodecharacterbyusingtheunichr()builtinfunction:
In[2]:a='aa'+unichr(170)+'bb'
In[3]:a
Out[3]:u'aa\xaabb'
In[6]:b=a.encode('utf8')
In[7]:b
Out[7]:'aa\xc2\xaabb'
In[8]:printb
aabb
GuidanceforuseofencodingsandunicodeIfyouareworkingwithamultibyte
characterset:
1. Convert/decodefromanexternalencodingtounicodeearly
(my_string.decode(encoding)).
2. Doyourworkinunicode.
3. Convert/encodetoanexternalencodinglate
(my_string.encode(encoding)).
Formoreinformation,see:
UnicodeInPython,CompletelyDemystifiedhttp://farmdev.com/talks/unicode/
PEP100:PythonUnicodeIntegration
http://www.python.org/dev/peps/pep0100/
InthePythonstandardlibrary:
codecsCodecregistryandbaseclasses
http://docs.python.org/2/library/codecs.html#modulecodecs
StandardEncodings
http://docs.python.org/2/library/codecs.html#standardencodings
Ifyouarereadingandwritingmultibytecharacterdatafromortoafile,thenlookatthe
Page28
APythonBook
codecs.open()inthecodecsmodule
http://docs.python.org/2/library/codecs.html#codecs.open.
HandlingmultibytecharactersetsinPython3iseasier,Ithink,butdifferent.Onehintis
tousetheencodingkeywordparametertotheopenbuiltinfunction.Hereisan
example:
deftest():
infile=open('infile1.txt','r',encoding='utf8')
outfile=open('outfile1.txt','w',encoding='utf8')
forlineininfile:
line=line.upper()
outfile.write(line)
infile.close()
outfile.close()
test()
1.4.4Dictionaries
Adictionaryisacollection,whosevaluesareaccessiblebykey.Itisacollectionof
namevaluepairs.
Theorderofelementsinadictionaryisundefined.But,wecaniterateover(1)thekeys,
(2)thevalues,and(3)theitems(keyvaluepairs)inadictionary.Wecansetthevalueof
akeyandwecangetthevalueassociatedwithakey.
Keysmustbeimmutableobjects:ints,strings,tuples,...
Literalsforconstructingdictionaries:
d1={}
d2={key1:value1,key2:value2,}
Constructorfordictionariesdict()canbeusedtocreateinstancesoftheclassdict.
Someexamples:
dict({'one':2,'two':3})
dict({'one':2,'two':3}.items())
dict({'one':2,'two':3}.iteritems())
dict(zip(('one','two'),(2,3)))
dict([['two',3],['one',2]])
dict(one=2,two=3)
dict([(['one','two'][i2],i)foriin(2,3)])
Foroperationsondictionaries,seehttp://docs.python.org/lib/typesmapping.htmloruse:
>>>help({})
Or:
Page29
APythonBook
>>>dir({})
IndexingAccessoradditemstoadictionarywiththeindexingoperator[].Example:
In[102]:dict1={}
In[103]:dict1['name']='dave'
In[104]:dict1['category']=38
In[105]:dict1
Out[105]:{'category':38,'name':'dave'}
Someoftheoperationsproducethekeys,thevalues,andtheitems(pairs)inadictionary.
Examples:
In[43]:d={'aa':111,'bb':222}
In[44]:d.keys()
Out[44]:['aa','bb']
In[45]:d.values()
Out[45]:[111,222]
In[46]:d.items()
Out[46]:[('aa',111),('bb',222)]
Wheniteratingoverlargedictionaries,usemethodsiterkeys(),itervalues(),
anditeritems().Example:
In[47]:
In[47]:d={'aa':111,'bb':222}
In[48]:forkeyind.iterkeys():
....:printkey
....:
....:
aa
bb
Totestfortheexistenceofakeyinadictionary,usetheinoperatororthe
mydict.has_key(k)method.Theinoperatorispreferred.Example:
>>>d={'tomato':101,'cucumber':102}
>>>k='tomato'
>>>kind
True
>>>d.has_key(k)
True
Youcanoftenavoidtheneedforatestbyusingmethodget.Example:
>>>d={'tomato':101,'cucumber':102}
>>>d.get('tomato',1)
101
>>>d.get('chard',1)
1
>>>ifd.get('eggplant')isNone:
...print'missing'
Page30
APythonBook
...
missing
Dictionary"view"objectsprovidedynamic(automaticallyupdated)viewsofthekeysor
thevaluesortheitemsinadictionary.Viewobjectsalsosupportsetoperations.Create
viewswithmydict.viewkeys(),mydict.viewvalues(),and
mydict.viewitems().See:
http://docs.python.org/2/library/stdtypes.html#dictionaryviewobjects.
Thedictionarysetdefaultmethodprovidesawaytogetthevalueassociatedwitha
keyfromadictionaryandtosetthatvalueifthekeyismissing.Example:
In[106]:a
Out[106]:{}
In[108]:a.setdefault('cc',33)
Out[108]:33
In[109]:a
Out[109]:{'cc':33}
In[110]:a.setdefault('cc',44)
Out[110]:33
In[111]:a
Out[111]:{'cc':33}
Exercises:
Writealiteralthatdefinesadictionaryusingbothstringliteralsandvariables
containingstrings.Solution:
>>>first='Dave'
>>>last='Kuhlman'
>>>name_dict={first:last,'Elvis':'Presley'}
>>>printname_dict
{'Dave':'Kuhlman','Elvis':'Presley'}
Writestatementsthatiterateover(1)thekeys,(2)thevalues,and(3)theitemsin
adictionary.(Note:Requiresintroductionoftheforstatement.)Solutions:
>>>d={'aa':111,'bb':222,'cc':333}
>>>forkeyind.keys():
...printkey
...
aa
cc
bb
>>>forvalueind.values():
...printvalue
...
111
333
222
>>>foritemind.items():
...printitem
Page31
APythonBook
...
('aa',111)
('cc',333)
('bb',222)
>>>forkey,valueind.items():
...printkey,'::',value
...
aa::111
cc::333
bb::222
Additionalnotesondictionaries:
Youcanuseiterkeys(),itervalues(),iteritems()toobtain
iteratorsoverkeys,values,anditems.
Adictionaryitselfisiterable:ititeratesoveritskeys.So,thefollowingtwolines
areequivalent:
forkinmyDict:printk
forkinmyDict.iterkeys():printk
Theinoperatortestsforakeyinadictionary.Example:
In[52]:mydict={'peach':'sweet','lemon':'tangy'}
In[53]:key='peach'
In[54]:ifkeyinmydict:
....:printmydict[key]
....:
sweet
1.4.5Files
Openafilewiththeopenfactorymethod.Example:
In[28]:f=open('mylog.txt','w')
In[29]:f.write('message#1\n')
In[32]:f.close()
In[33]:f=file('mylog.txt','r')
In[34]:forlineinf:
....:printline,
....:
message#1
message#2
message#3
In[35]:f.close()
Notes:
Usethe(builtin)open(path,mode)functiontoopenafileandcreateafile
object.Youcouldalsousefile(),butopen()isrecommended.
Page32
APythonBook
Afileobjectthatisopenforreadingatextfilesupportstheiteratorprotocoland,
therefore,canbeusedinaforstatement.Ititeratesoverthelinesinthefile.This
ismostlikelyonlyusefulfortextfiles.
openisafactorymethodthatcreatesfileobjects.Useittoopenfilesforreading,
writing,andappending.Examples:
infile=open('myfile.txt','r')#openforreading
outfile=open('myfile.txt','w')#openfor(over)
writing
log=open('myfile.txt','a')#openfor
appendingtoexistingcontent
Whenyouhavefinishedwithafile,closeit.Examples:
infile.close()
outfile.close()
Youcanalsousethewith:statementtoautomaticallyclosethefile.Example:
withopen('tmp01.txt','r')asinfile:
forxininfile:
printx,
Theaboveworksbecauseafileisacontextmanager:itobeysthecontext
managerprotocol.Afilehasmethods__enter__and__exit__,andthe
__exit__methodautomaticallyclosesthefileforus.Seethesectiononthe
with:statement.
Inordertoopenmultiplefiles,youcannestwith:statements,oruseasingle
with:statementwithmultiple"expressionastarget"clauses.Example:
deftest():
#
#usemultiplenestedwith:statements.
withopen('small_file.txt','r')asinfile:
withopen('tmp_outfile.txt','w')asoutfile:
forlineininfile:
outfile.write('line:%s'%
line.upper())
printinfile
printoutfile
#
#useasinglewith:statement.
withopen('small_file.txt','r')asinfile,\
open('tmp_outfile.txt','w')asoutfile:
forlineininfile:
outfile.write('line:%s'%line.upper())
printinfile
printoutfile
test()
fileisthefiletypeandcanbeusedasaconstructortocreatefileobjects.But,
Page33
APythonBook
openispreferred.
Linesreadfromatextfilehaveanewline.Stripitoffwithsomethinglike:
line.rstrip('\n').
Forbinaryfilesyoushouldaddthebinarymode,forexample:rb,wb.Formore
aboutmodes,seethedescriptionoftheopen()functionatBuiltinFunctions
http://docs.python.org/lib/builtinfuncs.html.
Learnmoreaboutfileobjectsandthemethodstheyprovideat:2.3.9FileObjects
http://docs.python.org/2/library/stdtypes.html#fileobjects.
Youcanalsoappendtoanexistingfile.Notethe"a"modeinthefollowingexample:
In[39]:f=open('mylog.txt','a')
In[41]:f.close()
In[42]:f=file('mylog.txt','r')
In[43]:forlineinf:
....:printline,
....:
message#1
message#2
message#3
message#4
In[44]:f.close()
Forbinaryfiles,add"b"tothemode.NotstrictlynecessaryonUNIX,butneededonMS
Windows.And,youwillwanttomakeyourcodeportableacrossplatforms.Example:
In[62]:importzipfile
In[63]:outfile=open('tmp1.zip','wb')
In[64]:zfile=zipfile.ZipFile(outfile,'w',zipfile.ZIP_DEFLATED)
In[65]:zfile.writestr('entry1','myheroeshavealwaysbeen
cowboys')
In[66]:zfile.writestr('entry2','andtheystillareitseems')
In[67]:zfile.writestr('entry3','sadlyinsearchofand')
In[68]:zfile.writestr('entry4','onstepinbackof')
In[69]:
In[70]:zfile.writestr('entry4','onestepinbackof')
In[71]:zfile.writestr('entry5','themselvesandtheirslowmoving
ways')
In[72]:zfile.close()
In[73]:outfile.close()
In[75]:
$
$unziplvtmp1.zip
Archive:tmp1.zip
LengthMethodSizeRatioDateTimeCRC32Name
34Defl:N366%05290817:04f6b7d921entry1
27Defl:N297%05290817:0710da8f3dentry2
22Defl:N249%05290817:073fd17fdaentry3
18Defl:N2011%05290817:08d55182e6entry4
Page34
APythonBook
19Defl:N2111%05290817:081a892acdentry4
37Defl:N395%05290817:09e213708centry5
1571698%6files
Exercises:
Readallofthelinesofafileintoalist.Printthe3rdand5thlinesinthefile/list.
Solution:
In[55]:f=open('tmp1.txt','r')
In[56]:lines=f.readlines()
In[57]:f.close()
In[58]:lines
Out[58]:['the\n','big\n','brown\n','dog\n',
'had\n','long\n','hair\n']
In[59]:printlines[2]
brown
In[61]:printlines[4]
had
Morenotes:
Stripnewlines(andotherwhitespace)fromastringwithmethodsstrip(),
lstrip(),andrstrip().
Getthecurrentpositionwithinafilebyusingmyfile.tell().
Setthecurrentpositionwithinafilebyusingmyfile.seek().Itmaybe
helpfultouseos.SEEK_CURandos.SEEK_END.Forexample:
f.seek(2,os.SEEK_CUR)advancesthepositionbytwo
f.seek(3,os.SEEK_END)setsthepositiontothethirdtolast.
f.seek(25)setsthepositionrelativetothebeginningofthefile.
1.4.6Otherbuiltintypes
OtherbuiltindatatypesaredescribedinsectionBuiltinTypes
http://docs.python.org/lib/types.htmlinthePythonstandarddocumentation.
1.4.6.1TheNonevalue/type
TheuniquevalueNoneisusedtoindicate"novalue","nothing","nonexistence",etc.
ThereisonlyoneNonevalue;inotherwords,it'sasingleton.
UseistotestforNone.Example:
>>>flag=None
>>>
>>>ifflagisNone:
...print'clear'
Page35
APythonBook
...
clear
>>>ifflagisnotNone:
...print'hello'
...
>>>
1.4.6.2Booleanvalues
TrueandFalsearethebooleanvalues.
Thefollowingvaluesalsocountasfalse,forexample,inanif:statement:False,
numericzero,None,theemptystring,anemptylist,anemptydictionary,anyempty
container,etc.Allothervalues,includingTrue,actastruevalues.
1.4.6.3Setsandfrozensets
Asetisanunorderedcollectionofimmutableobjects.Asetdoesnotcontainduplicates.
Setssupportseveralsetoperations,forexample:union,intersection,difference,...
Afrozensetislikeaset,exceptthatafrozensetisimmutable.Therefore,afrozensetis
hashableandcanbeusedasakeyinadictionary,anditcanbeaddedtoaset.
Createasetwiththesetconstructor.Examples:
>>>a=set()
>>>a
set([])
>>>a.add('aa')
>>>a.add('bb')
>>>a
set(['aa','bb'])
>>>b=set([11,22])
>>>b
set([11,22])
>>>c=set([22,33])
>>>b.union(c)
set([33,11,22])
>>>b.intersection(c)
set([22])
Formoreinformationonsets,see:SetTypesset,frozenset
http://docs.python.org/lib/typesset.html
1.5FunctionsandClassesAPreview
StructuredcodePythonprogramsaremadeupofexpressions,statements,functions,
classes,modules,andpackages.
Page36
APythonBook
Pythonobjectsarefirstclassobjects.
Expressionsareevaluated.
Statementsareexecuted.
Functions(1)areobjectsand(2)arecallable.
ObjectorientedprogramminginPython.Modeling"realworld"objects.(1)
Encapsulation;(2)datahiding;(3)inheritance.Polymorphism.
Classes(1)encapsulation;(2)datahiding;(3)inheritance.
Anoverviewofthestructureofatypicalclass:(1)methods;(2)theconstructor;(3)class
(static)variables;(4)super/subclasses.
1.6Statements
1.6.1Assignmentstatement
Formtarget=expression.
Possibletargets:
Identifier
TupleorlistCanbenested.Leftandrightsidesmusthaveequivalentstructure.
Example:
>>>x,y,z=11,22,33
>>>[x,y,z]=111,222,333
>>>a,(b,c)=11,(22,33)
>>>a,B=11,(22,33)
Thisfeaturecanbeusedtosimulateanenum:
In[22]:LITTLE,MEDIUM,LARGE=range(1,4)
In[23]:LITTLE
Out[23]:1
In[24]:MEDIUM
Out[24]:2
Subscriptionofasequence,dictionary,etc.Example:
In[10]:a=range(10)
In[11]:a
Out[11]:[0,1,2,3,4,5,6,7,8,9]
In[12]:a[3]='abc'
In[13]:a
Out[13]:[0,1,2,'abc',4,5,6,7,8,9]
In[14]:
In[14]:b={'aa':11,'bb':22}
In[15]:b
Page37
APythonBook
Out[15]:{'aa':11,'bb':22}
In[16]:b['bb']=1000
In[17]:b['cc']=2000
In[18]:b
Out[18]:{'aa':11,'bb':1000,'cc':2000}
AsliceofasequenceNotethatthesequencemustbemutable.Example:
In[1]:a=range(10)
In[2]:a
Out[2]:[0,1,2,3,4,5,6,7,8,9]
In[3]:a[2:5]=[11,22,33,44,55,66]
In[4]:a
Out[4]:[0,1,11,22,33,44,55,66,5,6,7,8,9]
AttributereferenceExample:
>>>classMyClass:
...pass
...
>>>anObj=MyClass()
>>>anObj.desc='pretty'
>>>printanObj.desc
pretty
Thereisalsoaugmentedassignment.Examples:
>>>index=0
>>>index+=1
>>>index+=5
>>>index+=f(x)
>>>index=1
>>>index*=3
Thingstonote:
Assignmenttoanamecreatesanewvariable(ifitdoesnotexistinthe
namespace)andabinding.Specifically,itbindsavaluetothenewname.Calling
afunctionalsodoesthistothe(formal)parameterswithinthelocalnamespace.
InPython,alanguagewithdynamictyping,thedatatypeisassociatedwiththe
value,notthevariable,asisthecaseinstaticallytypedlanguages.
Assignmentcanalsocausesharingofanobject.Example:
obj1=A()
obj2=obj1
Checktodeterminethatthesameobjectissharedwithid(obj)ortheis
operator.Example:
In[23]:a=range(10)
In[24]:a
Out[24]:[0,1,2,3,4,5,6,7,8,9]
In[25]:b=a
In[26]:b
Page38
APythonBook
Out[26]:[0,1,2,3,4,5,6,7,8,9]
In[27]:b[3]=333
In[28]:b
Out[28]:[0,1,2,333,4,5,6,7,8,9]
In[29]:a
Out[29]:[0,1,2,333,4,5,6,7,8,9]
In[30]:aisb
Out[30]:True
In[31]:printid(a),id(b)
3103792031037920
Youcanalsodomultipleassignmentinasinglestatement.Example:
In[32]:a=b=123
In[33]:a
Out[33]:123
In[34]:b
Out[34]:123
In[35]:
In[35]:
In[35]:a=b=[11,22]
In[36]:aisb
Out[36]:True
Youcaninterchange(swap)thevalueoftwovariablesusingassignmentand
packing/unpacking:
>>>a=111
>>>b=222
>>>a,b=b,a
>>>a
222
>>>b
111
1.6.2importstatement
Makemodule(orobjectsinthemodule)available.
Whatimportdoes:
Evaluatethecontentofamodule.
Likelytocreatevariablesinthelocal(module)namespace.
Evaluationofaspecificmoduleonlyhappensonceduringagivenrunofthe
program.Therefore,amoduleissharedacrossanapplication.
Amoduleisevaluatedfromtoptobottom.Laterstatementscanreplacevalues
createdearlier.Thisistrueoffunctionsandclasses,aswellas(other)variables.
Whichstatementsareevaluated?Assignment,class,def,...
Usethefollowingidiomtomakeamodulebothrunableandimportable:
if__name__=='__main__':
Page39
APythonBook
#importpdb;pdb.set_trace()
main()#or"test()"orsomeotherfunction
definedinmodule
Notes:
Theaboveconditionwillbetrueonlywhenthemoduleisrunasascriptand
willnotbetruewhenthemoduleisimported.
Thelinecontainingpdbcanbecopiedanyplaceinyourprogramand
uncommented,andthentheprogramwilldropintothePythondebugger
whenthatlocationisreached.
Whereimportlooksformodules:
sys.pathshowswhereitlooks.
Therearesomestandardplaces.
AddadditionaldirectoriesbysettingtheenvironmentvariablePYTHONPATH.
Youcanalsoaddpathsbymodifyingsys.path,forexample:
importsys
sys.path.insert(0,'/path/to/my/module')
Packagesneedafilenamed__init__.py.
ExtensionsTodeterminewhatextensionsimportlooksfor,do:
>>>importimp
>>>imp.get_suffixes()
[('.so','rb',3),('module.so','rb',3),('.py','U',
1),('.pyc','rb',2)]
Formsoftheimportstatement:
importANamesinthelocal(module)namespaceareaccessiblewiththedot
operator.
importAasBImportthemoduleA,butbindthemoduleobjecttothe
variableB.
importA1,A2Notrecommended
fromAimportB
fromAimportB1,B2
fromAimportBasC
fromAimport*Notrecommended:cluttersandmixesnamespaces.
fromA.BimportC(1)PossiblyimportobjectCfrommoduleBin
packageAor(2)possiblyimportmoduleCfromsubpackageBinpackageA.
importA.B.CToreferenceattributesinC,mustusefullyqualifiedname,
forexampleuseA.B.C.DtoreferenceDinsideofC.
Morenotesontheimportstatement:
TheimportstatementandpackagesAfilenamed__init__.pyisrequiredin
apackage.Thisfileisevaluatedthefirsttimeeitherthepackageisimportedora
Page40
APythonBook
fileinthepackageisimported.Question:Whatismadeavailablewhenyoudo
importaPackage?Answer:Allvariables(names)thatareglobalinsidethe
__init__.pymoduleinthatpackage.But,seenotesontheuseof__all__:
Theimportstatementhttp://docs.python.org/ref/import.html
Theuseofif__name__=="__main__":Makesamoduleboth
importableandexecutable.
UsingdotsintheimportstatementFromthePythonlanguagereferencemanual:
"Hierarchicalmodulenames:whenthemodulenamescontains
oneormoredots,themodulesearchpathiscarriedout
differently.Thesequenceofidentifiersuptothelastdotisused
tofindapackage;thefinalidentifieristhensearchedinside
thepackage.Apackageisgenerallyasubdirectoryofa
directoryonsys.paththathasafile__init__.py."
See:Theimportstatementhttp://docs.python.org/ref/import.html
Exercises:
Importamodulefromthestandardlibrary,forexamplere.
Importanelementfromamodulefromthestandardlibrary,forexampleimport
compilefromtheremodule.
CreateasimplePythonpackagewithasinglemoduleinit.Solution:
1. Createadirectorynamedsimplepackageinthecurrentdirectory.
2. Createan(empty)__init__.pyinthenewdirectory.
3. Createansimple.pyinthenewdirectory.
4. Addasimplefunctionnametest1insimple.py.
5. Importusinganyofthefollowing:
>>>importsimplepackage.simple
>>>fromsimplepackageimportsimple
>>>fromsimplepackage.simpleimporttest1
>>>fromsimplepackage.simpleimporttest1asmytest
1.6.3printstatement
printsendsoutputtosys.stdout.Itaddsanewline,unlessanextracommais
added.
Argumentstoprint:
MultipleitemsSeparatedbycommas.
Endwithcommatosuppresscarriagereturn.
Usestringformattingformorecontroloveroutput.
Alsoseevarious"prettyprinting"functionsandmethods,inparticular,pprint.
See3.27pprintDataprettyprinter
Page41
APythonBook
http://docs.python.org/lib/modulepprint.html.
StringformattingArgumentsareatuple.Reference:2.3.6.2StringFormatting
Operationshttp://docs.python.org/lib/typesseqstrings.html.
Canalsousesys.stdout.Notethatacarriagereturnisnotautomaticallyadded.
Example:
>>>importsys
>>>sys.stdout.write('hello\n')
ControllingthedestinationandformatofprintReplacesys.stdoutwithaninstance
ofanyclassthatimplementsthemethodwritetakingoneparameter.Example:
importsys
classWriter:
def__init__(self,file_name):
self.out_file=file(file_name,'a')
defwrite(self,msg):
self.out_file.write('[[%s]]'%msg)
defclose(self):
self.out_file.close()
deftest():
writer=Writer('outputfile.txt')
save_stdout=sys.stdout
sys.stdout=writer
print'hello'
print'goodbye'
writer.close()
#Showtheoutput.
tmp_file=file('outputfile.txt')
sys.stdout=save_stdout
content=tmp_file.read()
tmp_file.close()
printcontent
test()
Thereisanalternativeformoftheprintstatementthattakesafilelikeobject,in
particularanobjectthathasawritemethod.Forexample:
In[1]:outfile=open('tmp.log','w')
In[2]:print>>outfile,'Message#1'
In[5]:outfile.close()
In[6]:
In[6]:infile=open('tmp.log','r')
In[7]:forlineininfile:
...:print'Line:',line.rstrip('\n')
...:
Page42
APythonBook
Line:Message#1
Line:Message#2
Line:Message#3
In[8]:infile.close()
FuturedeprecationwarningThereisnoprintstatementinPython3.Thereisaprint
builtinfunction.
1.6.4if:elif:else:statement
Atemplatefortheif:statement:
ifcondition1:
statements
elifcondition2:
statements
elifcondition3:
statements
else:
statements
Theelifandelseclausesareoptional.
ConditionsExpressionsAnythingthatreturnsavalue.Comparewitheval()and
exec.
Truthvalues:
FalseFalse,None,numericzero,theemptystring,anemptycollection(list
ortupleordictionaryor...).
TrueTrueandeverythingelse.
Operators:
andandorNotethatbothandandordoshortcircuitevaluation.
not
isandisnotTheidenticalobject.Cf.aisbandid(a)==id(b).
UsefultotestforNone,forexample:
ifxisNone:
...
ifxisnotNone:
...
inandnotinCanbeusedtotestforexistenceofakeyinadictionaryorfor
thepresenceofavalueinacollection.
Theinoperatortestsforequality,notidentity.
Example:
>>>d={'aa':111,'bb':222}
>>>'aa'ind
Page43
APythonBook
True
>>>'aa'notind
False
>>>'xx'ind
False
Comparisonoperators,forexample==,!=,<,<=,...
Thereisanifexpression.Example:
>>>a='aa'
>>>b='bb'
>>>x='yes'ifa==belse'no'
>>>x
'no'
Notes:
Theelif:clausesandtheelse:clauseareoptional.
Theif:,elif:,andelse:clausesareallheaderlinesinthesensethatthey
areeachfollowedbyanindentedblockofcodeandeachoftheseheaderlines
endswithacolon.(Toputanemptyblockafteroneofthese,oranyother,
statementheaderline,usethepassstatement.It'seffectivelyanoop.)
Parenthesesaroundtheconditioninanif:orelif:arenotrequiredandare
consideredbadform,unlesstheconditionextendsovermultiplelines,inwhich
caseparenthesesarepreferredoveruseofalinecontinuationcharacter(backslash
attheendoftheline).
Exercises:
Writeanifstatementwithanandoperator.
Writeanifstatementwithanoroperator.
Writeanifstatementcontainingbothandandoroperators.
1.6.5for:statement
Iterateoverasequenceoran"iterable"object.
Form:
forxiny:
block
IteratorSomenotesonwhatitmeanstobeiterable:
Aniterableissomethingthatcanbeusedinaniteratorcontext,forexample,ina
for:statement,inalistcomprehension,andinageneratorexpression.
Sequencesandcontainersareiterable.Examples:tuples,lists,strings,
dictionaries.
Instancesofclassesthatobeytheiteratorprotocolareiterable.See
Page44
APythonBook
http://docs.python.org/lib/typeiter.html.
Wecancreateaniteratorobjectwithbuiltinfunctionssuchasiter()and
enumerate().SeeBuiltinFunctions
http://docs.python.org/lib/builtinfuncs.htmlinthePythonstandardlibrary
reference.
Functionsthatusetheyieldstatement,produceaniterator,althoughit'sactually
calledagenerator.
Aniterableimplementstheiteratorinterfaceandsatisfiestheiteratorprotocol.
Theiteratorprotocol:__iter__()andnext()methods.See2.3.5Iterator
Types(http://docs.python.org/lib/typeiter.html).
Testingfor"iterability":
Ifyoucanuseanobjectinafor:statement,it'siterable.
Iftheexpresioniter(obj)doesnotproduceaTypeErrorexception,it's
iterable.
Somewaystoproduceiterators:
iter()andenumerate()See:
http://docs.python.org/lib/builtinfuncs.html.
some_dict.iterkeys(),some_dict.itervalues(),
some_dict.iteritems().
Useasequenceinaniteratorcontext,forexampleinaforstatement.Lists,
tuples,dictionaries,andstringscanbeusedinaniteratorcontexttoproducean
iterator.
GeneratorexpressionsLatestPythononly.Syntacticallylikelist
comprehensions,but(1)surroundedbyparenthesesinsteadofsquarebracketsand
(2)uselazyevaluation.
AclassthatimplementstheiteratorprotocolExample:
classA(object):
def__init__(self):
self.data=[11,22,33]
self.idx=0
def__iter__(self):
returnself
defnext(self):
ifself.idx<len(self.data):
x=self.data[self.idx]
self.idx+=1
returnx
else:
raiseStopIteration
deftest():
a=A()
forxina:
Page45
APythonBook
printx
test()
NotethattheiteratorprotocolchangesinPython3.
Afunctioncontainingayieldstatement.See:
Yieldexpressions
http://docs.python.org/2/reference/expressions.html#yieldexpressions
Theyieldstatement
http://docs.python.org/2/reference/simple_stmts.html#theyieldstatement
AlsoseeitertoolsmoduleinthePythonstandardlibraryformuchmorehelp
withiterators:itertoolsFunctionscreatingiteratorsforefficientlooping
http://docs.python.org/2/library/itertools.html#moduleitertools
Thefor:statementcanalsodounpacking.Example:
In[25]:items=['apple','banana','cherry','date']
In[26]:foridx,iteminenumerate(items):
....:print'%d.%s'%(idx,item,)
....:
0.apple
1.banana
2.cherry
3.date
Theforstatementcanalsohaveanoptionalelse:clause.Theelse:clauseis
executediftheforstatementcompletesnormally,thatisifabreakstatementisnot
executed.
Helpfulfunctionswithfor:
enumerate(iterable)Returnsaniterablethatproducespairs(tuples)
containingcount(index)andvalue.Example:
>>>foridx,valueinenumerate([11,22,33]):
...printidx,value
...
011
122
233
range([start,]stop[,step])andxrange([start,]stop[,
step]).
ListcomprehensionsSincelistcomprehensionscreatelists,theyareusefulinfor
statements,although,whenthenumberofelementsislarge,youshouldconsiderusinga
generatorexpressioninstead.Alistcomprehensionlooksabitlikeafor:statement,but
isinsidesquarebrackets,anditisanexpression,notastatement.Twoforms(among
others):
[f(x)forxiniterable]
Page46
APythonBook
[f(x)forxiniterableift(x)]
GeneratorexpressionsAgeneratorexpressionlookssimilartoalistcomprehension,
exceptthatitissurroundedbyparenthesesratherthansquarebrackets.Example:
In[28]:items=['apple','banana','cherry','date']
In[29]:gen1=(item.upper()foriteminitems)
In[30]:forxingen1:
....:print'x:',x
....:
x:APPLE
x:BANANA
x:CHERRY
x:DATE
Exercises:
Writealistcomprehensionthatreturnsallthekeysinadictionarywhose
associatedvaluesaregreaterthanzero.
Thedictionary:{'aa':11,'cc':33,'dd':55,'bb':22}
Solution:[x[0]forxinmy_dict.iteritems()ifx[1]>
0]
Writealistcomprehensionthatproducesevenintegersfrom0to10.Useafor
statementtoiterateoverthosevalues.Solution:
forxin[yforyinrange(10)ify%2==0]:
print'x:%s'%x
Writealistcomprehensionthatiteratesovertwolistsandproducesallthe
combinationsofitemsfromthelists.Solution:
In[19]:a=range(4)
In[20]:b=[11,22,33]
In[21]:a
Out[21]:[0,1,2,3]
In[22]:b
Out[22]:[11,22,33]
In[23]:c=[(x,y)forxinaforyinb]
In[24]:printc
[(0,11),(0,22),(0,33),(1,11),(1,22),(1,33),
(2,11),(2,22),(2,33),(3,11),(3,22),(3,33)]
But,notethatinthepreviousexercise,ageneratorexpressionwouldoftenbebetter.A
generatorexpressionislikealistcomprehension,exceptthat,insteadofcreatingthe
entirelist,itproducesageneratorthatcanbeusedtoproduceeachoftheelements.
Thebreakandcontinuestatementsareoftenusefulinaforstatement.Seecontinue
andbreakstatements
Theforstatementcanalsohaveanoptionalelse:clause.Theelse:clauseis
executediftheforstatementcompletesnormally,thatisifabreakstatementisnot
executed.Example:
Page47
APythonBook
foritemindata1:
ifitem>100:
value1=item
break
else:
value1='notfound'
print'value1:',value1
Whenrun,itprints:
value1:notfound
1.6.6while:statement
Form:
whilecondition:
block
Thewhile:statementisnotoftenusedinPythonbecausethefor:statementis
usuallymoreconvenient,moreidiomatic,andmorePythonic.
Exercises:
Writeawhilestatementthatprintsintegersfromzeroto5.Solution:
count=0
whilecount<5:
count+=1
printcount
Thebreakandcontinuestatementsareoftenusefulinawhilestatement.See
continueandbreakstatements
Thewhilestatementcanalsohaveanoptionalelse:clause.Theelse:clauseis
executedifthewhilestatementcompletesnormally,thatisifabreakstatementisnot
executed.
1.6.7continueandbreakstatements
Thebreakstatementexitsfromaloop.
Thecontinuestatementcausesexecutiontoimmediatelycontinueatthestartofthe
loop.
Canbeusedinfor:andwhile:.
Whenthefor:statementorthewhile:statementhasanelse:clause,theblockin
theelse:clauseisexecutedonlyifabreakstatementisnotexecuted.
Exercises:
Page48
APythonBook
Usingbreak,writeawhilestatementthatprintsintegersfromzeroto5.
Solution:
count=0
whileTrue:
count+=1
ifcount>5:
break
printcount
Notes:
Aforstatementthatusesrange()orxrange()wouldbebetterthana
whilestatementforthisuse.
Usingcontinue,writeawhilestatementthatprocessesonlyevenintegers
from0to10.Note:%isthemodulooperator.Solution:
count=0
whilecount<10:
count+=1
ifcount%2==0:
continue
printcount
1.6.8try:except:statement
Exceptionsareasystematicandconsistentwayofprocessingerrorsand"unusual"events
inPython.
CaughtanduncaughtexceptionsUncaughtexceptionsterminateaprogram.
Thetry:statementcatchesanexception.
AlmostallerrorsinPythonareexceptions.
Evaluation(executionmodel)ofthetrystatementWhenanexceptionoccursinthe
tryblock,evenifinsideanestedfunctioncall,executionofthetryblockendsandthe
exceptclausesaresearchedforamatchingexception.
TracebacksAlsoseethetracebackmodule:
http://docs.python.org/lib/moduletraceback.html
Exceptionsareclasses.
Exceptionclassessubclassing,args.
Anexceptionclassinanexcept:clausecatchesinstancesofthatexceptionclassand
allsubclasses,butnotsuperclasses.
BuiltinexceptionclassesSee:
Moduleexceptions.
Page49
APythonBook
Builtinexceptionshttp://docs.python.org/lib/moduleexceptions.html.
UserdefinedexceptionclassessubclassesofException.
Example:
try:
raiseRuntimeError('thissillyerror')
exceptRuntimeError,exp:
print"[[[%s]]]"%exp
Reference:http://docs.python.org/lib/moduleexceptions.html
Youcanalsogettheargumentspassedtotheconstructorofanexceptionobject.Inthe
aboveexample,thesewouldbe:
exp.args
Whywouldyoudefineyourownexceptionclass?Oneanswer:Youwantauserofyour
codetocatchyourexceptionandnoothers.
Catchinganexceptionbyexceptionclasscatchesexceptionsofthatclassandallits
subclasses.So:
exceptSomeExceptionClass,exp:
matchesandcatchesanexceptionifSomeExceptionClassistheexceptionclassorabase
class(superclass)oftheexceptionclass.Theexceptionobject(usuallyaninstanceof
someexceptionclass)isboundtoexp.
Amore"modern"syntaxis:
exceptSomeExceptionClassasexp:
So:
classMyE(ValueError):
pass
try:
raiseMyE()
exceptValueError:
print'caughtexception'
willprint"caughtexception",becauseValueErrorisabaseclassofMyE.
Alsoseetheentriesfor"EAFP"and"LBYL"inthePythonglossary:
http://docs.python.org/3/glossary.html.
Exercises:
Writeaverysimple,emptyexceptionsubclass.Solution:
classMyE(Exception):
Page50
APythonBook
pass
Writeatry:except:statementthatraisesyourexceptionandalsocatchesit.
Solution:
try:
raiseMyE('hellotheredave')
exceptMyE,e:
printe
1.6.9raisestatement
Throworraiseanexception.
Forms:
raiseinstance
raiseMyExceptionClass(value)preferred.
raiseMyExceptionClass,value
Theraisestatementtakes:
An(instanceof)abuiltinexceptionclass.
AninstanceofclassExceptionor
AninstanceofabuiltinsubclassofclassExceptionor
AninstanceofauserdefinedsubclassofclassExceptionor
Oneoftheaboveclassesand(optionally)avalue(forexample,astringora
tuple).
Seehttp://docs.python.org/ref/raise.html.
Foralistofbuiltinexceptions,seehttp://docs.python.org/lib/moduleexceptions.html.
Thefollowingexampledefinesanexceptionsubclassandthrowsaninstanceofthat
subclass.Italsoshowshowtopassandcatchmultipleargumentstotheexception:
classNotsobadError(Exception):
pass
deftest(x):
try:
ifx==0:
raiseNotsobadError('amoderatelybaderror','nottoo
bad')
exceptNotsobadError,e:
print'Errorargs:%s'%(e.args,)
test(0)
Whichprintsoutthefollowing:
Errorargs:('amoderatelybaderror','nottoobad')
Page51
APythonBook
Notes:
Inordertopassinmultipleargumentswiththeexception,weuseatuple,orwe
passmultipleargumentstotheconstructor.
Thefollowingexampledoesasmallamountofprocessingofthearguments:
classNotsobadError(Exception):
"""Anexceptionclass.
"""
defget_args(self):
return'::::'.join(self.args)
deftest(x):
try:
ifx==0:
raiseNotsobadError('amoderatelybaderror','nottoo
bad')
exceptNotsobadError,e:
print'Errorargs:{{{%s}}}'%(e.get_args(),)
test(0)
1.6.10with:statement
Thewithstatementenablesustouseacontextmanager(anyobjectthatsatisfiesthe
contextmanagerprotocol)toaddcodebefore(onentryto)andafter(onexitfrom)a
blockofcode.
1.6.10.1Writingacontextmanager
Acontextmanagerisaninstanceofaclassthatsatisfiesthisinterface:
classContext01(object):
def__enter__(self):
pass
def__exit__(self,exc_type,exc_value,traceback):
pass
Hereisanexamplethatusestheabovecontextmanager:
classContext01(object):
def__enter__(self):
print'in__enter__'
return'somevalueorother'#usuallywewanttoreturn
self
def__exit__(self,exc_type,exc_value,traceback):
print'in__exit__'
Notes:
Page52
APythonBook
The__enter__methodiscalledbeforeourblockofcodeisentered.
Usually,butnotalways,wewillwantthe__enter__methodtoreturnself,
thatis,theinstanceofourcontextmanagerclass.Wedothissothatwecanwrite:
withMyContextManager()asobj:
pass
andthenusetheinstance(objinthiscase)inthenestedblock.
The__exit__methodiscalledwhenourblockofcodeisexitedeither
normallyorbecauseofanexception.
Ifanexceptionissupplied,andthemethodwishestosuppresstheexception(i.e.,
preventitfrombeingpropagated),itshouldreturnatruevalue.Otherwise,the
exceptionwillbeprocessednormallyuponexitfromthismethod.
Iftheblockexitsnormally,thevalueofexc_type,exc_value,and
tracebackwillbeNone.
Formoreinformationonthewith:statement,seeContextManagerTypes
http://docs.python.org/2/library/stdtypes.html#contextmanagertypes.
Seemodulecontextlibforstrangewaysofwritingcontextmanagers:
http://docs.python.org/2/library/contextlib.html#modulecontextlib
1.6.10.2Usingthewith:statement
Hereareexamples:
#example1
withContext01():
print'inbody'
#example2
withContext02()asa_value:
print'inbody'
print'a_value:"%s"'%(a_value,)
a_value.some_method_in_Context02()
#example3
withopen(infilename,'r')asinfile,open(outfilename,'w')as
outfile:
forlineininfile:
line=line.rstrip()
outfile.write('%s\n'%line.upper())
Notes:
Intheformwith...asval,thevaluereturnedbythe__enter__
methodisassignedtothevariable(valinthiscase).
Inordertousemorethanonecontextmanager,youcannestwith:statements,
orseparateusesofofthecontextmanagerswithcommas,whichisusually
Page53
APythonBook
preferred.Seeexample3above.
1.6.11del
Thedelstatementcanbeusedto:
Removenamesfromnamespace.
Removeitemsfromacollection.
Ifnameislistedinaglobalstatement,thendelremovesnamefromtheglobal
namespace.
Namescanbea(nested)list.Examples:
>>>dela
>>>dela,b,c
Wecanalsodeleteitemsfromalistordictionary(andperhapsfromotherobjectsthatwe
cansubscript).Examples:
In[9]:d={'aa':111,'bb':222,'cc':333}
In[10]:printd
{'aa':111,'cc':333,'bb':222}
In[11]:deld['bb']
In[12]:printd
{'aa':111,'cc':333}
In[13]:
In[13]:a=[111,222,333,444]
In[14]:printa
[111,222,333,444]
In[15]:dela[1]
In[16]:printa
[111,333,444]
And,wecandeleteanattributefromaninstance.Example:
In[17]:classA:
....:pass
....:
In[18]:a=A()
In[19]:a.x=123
In[20]:dir(a)
Out[20]:['__doc__','__module__','x']
In[21]:printa.x
123
In[22]:dela.x
In[23]:dir(a)
Out[23]:['__doc__','__module__']
In[24]:printa.x
exceptions.AttributeErrorTraceback(mostrecentcalllast)
Page54
APythonBook
/home/dkuhlman/a1/Python/Test/<console>
AttributeError:Ainstancehasnoattribute'x'
1.6.12casestatement
ThereisnocasestatementinPython.Usetheif:statementwithasequenceofelif:
clauses.Or,useadictionaryoffunctions.
1.7Functions,Modules,Packages,andDebugging
1.7.1Functions
1.7.1.1Thedefstatement
Thedefstatementisusedtodefinefunctionsandmethods.
Thedefstatementisevaluated.Itproducesafunction/method(object)andbindsittoa
variableinthecurrentnamespace.
Althoughthedefstatementisevaluated,thecodeinitsnestedblockisnotexecuted.
Therefore,manyerrorsmaynotbedetecteduntileachandeverypaththroughthatcodeis
tested.Recommendations:(1)UseaPythoncodechecker,forexampleflake8or
pylint;(2)DothoroughtestingandusethePythonunittestframework.Pythonic
wisdom:Ifit'snottested,it'sbroken.
1.7.1.2Returningvalues
Thereturnstatementisusedtoreturnvaluesfromafunction.
Thereturnstatementtakeszeroormorevalues,separatedbycommas.Usingcommas
actuallyreturnsasingletuple.
ThedefaultvalueisNone.
Toreturnmultiplevalues,useatupleorlist.Don'tforgetthat(assignment)unpacking
canbeusedtocapturemultiplevalues.Returningmultipleitemsseparatedbycommasis
equivalenttoreturningatuple.Example:
In[8]:deftest(x,y):
...:returnx*3,y*4
...:
In[9]:a,b=test(3,4)
In[10]:printa
9
Page55
APythonBook
In[11]:printb
16
1.7.1.3Parameters
DefaultvaluesExample:
In[53]:deft(max=5):
....:forvalinrange(max):
....:printval
....:
....:
In[54]:t(3)
0
1
2
In[55]:t()
0
1
2
3
4
Givingaparameteradefaultvaluemakesthatparameteroptional.
Note:Ifafunctionhasaparameterwithadefaultvalue,thenall"normal"arguments
mustproceedtheparameterswithdefaultvalues.Morecompletely,parametersmustbe
givenfromlefttorightinthefollowingorder:
1. Normalarguments.
2. Argumentswithdefaultvalues.
3. Argumentlist(*args).
4. Keywordarguments(**kwargs).
Listparameters*args.It'satuple.
Keywordparameters**kwargs.It'sadictionary.
1.7.1.4Arguments
Whencallingafunction,valuesmaybepassedtoafunctionwithpositionalargumentsor
keywordarguments.
Positionalargumentsmustplacedbefore(totheleftof)keywordarguments.
Passingliststoafunctionasmultipleargumentssome_func(*aList).Effectively,
thissyntaxcausesPythontounrollthearguments.Example:
deffn1(*args,**kwargs):
fn2(*args,**kwargs)
Page56
APythonBook
1.7.1.5Localvariables
CreatinglocalvariablesAnybindingoperationcreatesalocalvariable.Examplesare
(1)parametersofafunction;(2)assignmenttoavariableinafunction;(3)theimport
statement;(4)etc.Contrastwithaccessingavariable.
VariablelookupTheLGB/LEGBruleThelocal,enclosing,global,builtinscopes
aresearchedinthatorder.See:http://www.python.org/dev/peps/pep0227/
TheglobalstatementInsideafunction,wemustuseglobalwhenwewanttoset
thevalueofaglobalvariable.Example:
deffn():
globalSome_global_variable,Another_global_variable
Some_global_variable='hello'
...
1.7.1.6Otherthingstoknowaboutfunctions
FunctionsarefirstclassYoucanstoretheminastructure,passthemtoa
function,andreturnthemfromafunction.
Functioncallscantakekeywordarguments.Example:
>>>test(size=25)
Formalparameterstoafunctioncanhavedefaultvalues.Example:
>>>deftest(size=0):
...
Donotusemutableobjectsasdefaultvalues.
Youcan"capture"remainingargumentswith*args,and**kwargs.(Spelling
isnotsignificant.)Example:
In[13]:deftest(size,*args,**kwargs):
....:printsize
....:printargs
....:printkwargs
....:
....:
In[14]:test(32,'aa','bb',otherparam='xyz')
32
('aa','bb')
{'otherparam':'xyz'}
Normalargumentsmustcomebeforedefaultargumentswhichmustcomebefore
keywordarguments.
Afunctionthatdoesnotexplicitlyreturnavalue,returnsNone.
Inordertosetthevalueofaglobalvariable,declarethevariablewithglobal.
Exercises:
Page57
APythonBook
Writeafunctionthattakesasingleargument,printsthevalueoftheargument,
andreturnstheargumentasastring.Solution:
>>>deft(x):
...print'x:%s'%x
...return'[[%s]]'%x
...
>>>t(3)
x:3
'[[3]]'
Writeafunctionthattakesavariablenumberofargumentsandprintsthemall.
Solution:
>>>deft(*args):
...forarginargs:
...print'arg:%s'%arg
...
>>>t('aa','bb','cc')
arg:aa
arg:bb
arg:cc
Writeafunctionthatprintsthenamesandvaluesofkeywordargumentspassedto
it.Solution:
>>>deft(**kwargs):
...forkeyinkwargs.keys():
...print'key:%svalue:%s'%(key,
kwargs[key],)
...
>>>t(arg1=11,arg2=22)
key:arg1value:11
key:arg2value:22
1.7.1.7Globalvariablesandtheglobalstatement
Bydefault,assignmentinafunctionormethodcreateslocalvariables.
Reference(notassignment)toavariable,accessesalocalvariableifithasalreadybeen
created,elseaccessesaglobalvariable.
Inordertoassignavaluetoaglobalvariable,declarethevariableasglobalatthe
beginningofthefunctionormethod.
Ifinafunctionormethod,youbothreferenceandassigntoavariable,thenyoumust
either:
1. Assigntothevariablefirst,or
2. Declarethevariableasglobal.
Theglobalstatementdeclaresoneormorevariables,separatedbycommas,tobe
global.
Page58
APythonBook
Someexamples:
In[1]:
In[1]:X=3
In[2]:deft():
...:printX
...:
In[3]:
In[3]:t()
3
In[4]:defs():
...:X=4
...:
In[5]:
In[5]:
In[5]:s()
In[6]:t()
3
In[7]:X=1
In[8]:defu():
...:globalX
...:X=5
...:
In[9]:
In[9]:u()
In[10]:t()
5
In[16]:defv():
....:x=X
....:X=6
....:returnx
....:
In[17]:
In[17]:v()
Traceback(mostrecentcalllast):
File"<ipythonconsole>",line1,in<module>
File"<ipythonconsole>",line2,inv
UnboundLocalError:localvariable'X'referencedbeforeassignment
In[18]:defw():
....:globalX
....:x=X
....:X=7
....:returnx
....:
In[19]:
In[19]:w()
Out[19]:5
In[20]:X
Out[20]:7
Page59
APythonBook
1.7.1.8Docstringsforfunctions
Adddocstringsasatriplequotedstringbeginningwiththefirstlineofafunctionor
method.Seeepydocforasuggestedformat.
1.7.1.9Decoratorsforfunctions
Adecoratorperformsatransformationonafunction.Examplesofdecoratorsthatare
builtinfunctionsare:@classmethod,@staticmethod,and@property.See:
http://docs.python.org/2/library/functions.html#builtinfunctions
Adecoratorisappliedusingthe"@"characteronalineimmediatelypreceedingthe
functiondefinitionheader.Examples:
classSomeClass(object):
@classmethod
defHelloClass(cls,arg):
pass
##HelloClass=classmethod(HelloClass)
@staticmethod
defHelloStatic(arg):
pass
##HelloStatic=staticmethod(HelloStatic)
#
#Define/implementadecorator.
defwrapper(fn):
definner_fn(*args,**kwargs):
print'>>'
result=fn(*args,**kwargs)
print'<<'
returnresult
returninner_fn
#
#Applyadecorator.
@wrapper
deffn1(msg):
pass
##fn1=wrapper(fn1)
Notes:
Thedecoratorform(withthe"@"character)isequivalenttotheform
(commentedout)thatcallsthedecoratorfunctionexplicitly.
Theuseofclassmethodsandstaticmethodwillbeexplainedlaterinthe
sectiononobjectorientedprogramming.
Adecoratorisimplementedasafunction.Therefore,tolearnaboutsomespecific
Page60
APythonBook
decorator,youshouldsearchforthedocumentationonortheimplementationof
thatfunction.Rememberthatinordertouseafunction,itmustbedefinedinthe
currentmoduleorimportedbythecurrentmoduleorbeabuiltin.
Theformthatexplicitlycallsthedecoratorfunction(commentedoutinthe
exampleabove)isequivalenttotheformusingthe"@"character.
1.7.2lambda
Usealambda,asaconvenience,whenyouneedafunctionthatboth:
isanonymous(doesnotneedaname)and
containsonlyanexpressionandnostatements.
Example:
In[1]:fn=lambdax,y,z:(x**2)+(y*2)+z
In[2]:fn(4,5,6)
Out[2]:32
In[3]:
In[3]:format=lambdaobj,category:'The"%s"isa"%s".'%(obj,
category,)
In[4]:format('pinetree','conifer')
Out[4]:'The"pinetree"isa"conifer".'
Alambdacantakemultipleargumentsandcanreturn(likeafunction)multiplevalues.
Example:
In[79]:a=lambdax,y:(x*3,y*4,(x,y))
In[80]:
In[81]:a(3,4)
Out[81]:(9,16,(3,4))
Suggestion:Insomecases,alambdamaybeusefulasaneventhandler.
Example:
classTest:
def__init__(self,first='',last=''):
self.first=first
self.last=last
deftest(self,formatter):
"""
Testforlambdas.
formatterisafunctiontaking2arguments,firstandlast
names.Itshouldreturntheformattedname.
"""
msg='Mynameis%s'%(formatter(self.first,self.last),)
printmsg
deftest():
t=Test('Dave','Kuhlman')
Page61
APythonBook
t.test(lambdafirst,last:'%s%s'%(first,last,))
t.test(lambdafirst,last:'%s,%s'%(last,first,))
test()
Alambdaenablesustodefine"functions"wherewedonotneednamesforthose
functions.Example:
In[45]:a=[
....:lambdax:x,
....:lambdax:x*2,
....:]
In[46]:
In[46]:a
Out[46]:[<function__main__.<lambda>>,<function__main__.<lambda>>]
In[47]:a[0](3)
Out[47]:3
In[48]:a[1](3)
Out[48]:6
Reference:http://docs.python.org/2/reference/expressions.html#lambda
1.7.3Iteratorsandgenerators
Concepts:
iterator
Anditeratorissomethingthatsatisfiestheiteratorprotocol.Clue:Ifit'saniterator,
youcanuseitinafor:statement.
generator
Ageneratorisaclassorfunctionthatimplementsaniterator,i.e.thatimplementsthe
iteratorprotocol.
theiteratorprotocol
Anobjectsatisfiestheiteratorprotocolifitdoesthefollowing:
Itimplementsa__iter__method,whichreturnsaniteratorobject.
Itimplementsanextfunction,whichreturnsthenextitemfromthe
collection,sequence,stream,etcofitemstobeiteratedover
ItraisestheStopIterationexceptionwhentheitemsareexhaustedand
thenext()methodiscalled.
yield
Theyieldstatementenablesustowritefunctionsthataregenerators.Such
functionsmaybesimilartocoroutines,sincetheymay"yield"multipletimesandare
resumed.
Page62
APythonBook
Formoreinformationoniterators,seethesectiononiteratortypesinthePythonLibrary
Referencehttp://docs.python.org/2/library/stdtypes.html#iteratortypes.
Formoreontheyieldstatement,see:
http://docs.python.org/2/reference/simple_stmts.html#theyieldstatement
Actually,yieldisanexpression.Formoreonyieldexpressionsandonthenext()
andsend()generatormethods,aswellasothers,see:Yieldexpression
http://docs.python.org/2/reference/expressions.html#yieldexpressionsinthePython
languagereferencemanual.
Afunctionormethodcontainingayieldstatementimplementsagenerator.Addingthe
yieldstatementtoafunctionormethodturnsthatfunctionormethodintoonewhich,
whencalled,returnsagenerator,i.e.anobjectthatimplementstheiteratorprotocol.
Agenerator(afunctioncontainingyield)providesaconvenientwaytoimplementa
filter.But,alsoconsider:
Thefilter()builtinfunction
Listcomprehensionswithanifclause
Hereareafewexamples:
defsimplegenerator():
yield'aaa'#Note1
yield'bbb'
yield'ccc'
deflist_tripler(somelist):
foriteminsomelist:
item*=3
yielditem
deflimit_iterator(somelist,max):
foriteminsomelist:
ifitem>max:
return#Note2
yielditem
deftest():
print'1.',''*30
it=simplegenerator()
foriteminit:
printitem
print'2.',''*30
alist=range(5)
it=list_tripler(alist)
foriteminit:
printitem
print'3.',''*30
alist=range(8)
Page63
APythonBook
it=limit_iterator(alist,4)
foriteminit:
printitem
print'4.',''*30
it=simplegenerator()
try:
printit.next()#Note3
printit.next()
printit.next()
printit.next()
exceptStopIteration,exp:#Note4
print'reachedendofsequence'
test()
Notes:
1. Theyieldstatementreturnsavalue.Whenthenextitemisrequestedandthe
iteratoris"resumed",executioncontinuesimmediatelyaftertheyield
statement.
2. Wecanterminatethesequencegeneratedbyaniteratorbyusingareturn
statementwithnovalue.
3. Toresumeagenerator,usethegenerator'snext()orsend()methods.
send()islikenext(),butprovidesavaluetotheyieldexpression.
4. Wecanalternativelyobtaintheitemsinasequencebycallingtheiterator's
next()method.Sinceaniteratorisafirstclassobject,wecansaveitinadata
structureandcanpassitaroundforuseatdifferentlocationsandtimesinour
program.
1. Whenaniteratorisexhaustedorempty,itthrowstheStopIteration
exception,whichwecancatch.
Andhereistheoutputfromrunningtheaboveexample:
$pythontest_iterator.py
1.
aaa
bbb
ccc
2.
0
3
6
9
12
3.
0
1
2
3
Page64
APythonBook
4
4.
aaa
bbb
ccc
reachedendofsequence
Aninstanceofaclasswhichimplementsthe__iter__method,returninganiterator,is
iterable.Forexample,itcanbeusedinaforstatementorinalistcomprehension,orin
ageneratorexpression,orasanargumenttotheiter()builtinmethod.But,notice
thattheclassmostlikelyimplementsageneratormethodwhichcanbecalleddirectly.
ExamplesThefollowingcodeimplementsaniteratorthatproducesalltheobjectsina
treeofobjects:
classNode:
def__init__(self,data,children=None):
self.initlevel=0
self.data=data
ifchildrenisNone:
self.children=[]
else:
self.children=children
defset_initlevel(self,initlevel):self.initlevel=initlevel
defget_initlevel(self):returnself.initlevel
defaddchild(self,child):
self.children.append(child)
defget_data(self):
returnself.data
defget_children(self):
returnself.children
defshow_tree(self,level):
self.show_level(level)
print'data:%s'%(self.data,)
forchildinself.children:
child.show_tree(level+1)
defshow_level(self,level):
print''*level,
#
#Generatormethod#1
#Thisgeneratorturnsinstancesofthisclassintoiterable
objects.
#
defwalk_tree(self,level):
yield(level,self,)
forchildinself.get_children():
forlevel1,tree1inchild.walk_tree(level+1):
yieldlevel1,tree1
def__iter__(self):
returnself.walk_tree(self.initlevel)
Page65
APythonBook
#
#Generatormethod#2
#Thisgeneratorusesasupportfunction(walk_list)whichcalls
#thisfunctiontorecursivelywalkthetree.
#Ifeffect,thisiteratesoverthesupportfunction,which
#iteratesoverthisfunction.
#
defwalk_tree(tree,level):
yield(level,tree)
forchildinwalk_list(tree.get_children(),level+1):
yieldchild
defwalk_list(trees,level):
fortreeintrees:
fortreeinwalk_tree(tree,level):
yieldtree
#
#Generatormethod#3
#Thisgeneratorislikemethod#2,butcallsitself(asan
iterator),
#ratherthancallingasupportfunction.
#
defwalk_tree_recur(tree,level):
yield(level,tree,)
forchildintree.get_children():
forlevel1,tree1inwalk_tree_recur(child,level+1):
yield(level1,tree1,)
defshow_level(level):
print''*level,
deftest():
a7=Node('777')
a6=Node('666')
a5=Node('555')
a4=Node('444')
a3=Node('333',[a4,a5])
a2=Node('222',[a6,a7])
a1=Node('111',[a2,a3])
initLevel=2
a1.show_tree(initLevel)
print'='*40
forlevel,iteminwalk_tree(a1,initLevel):
show_level(level)
print'item:',item.get_data()
print'='*40
forlevel,iteminwalk_tree_recur(a1,initLevel):
show_level(level)
Page66
APythonBook
print'='*40
a1.set_initlevel(initLevel)
forlevel,itemina1:
show_level(level)
iter1=iter(a1)
printiter1
printiter1.next()
printiter1.next()
printiter1.next()
printiter1.next()
printiter1.next()
printiter1.next()
printiter1.next()
##printiter1.next()
returna1
test()
Notes:
AninstanceofclassNodeis"iterable".Itcanbeuseddirectlyinafor
statement,alistcomprehension,etc.So,forexample,whenaninstanceofNode
isusedinaforstatement,itproducesaniterator.
WecouldalsocalltheNode.walk_methoddirectlytoobtainaniterator.
MethodNode.walk_treeandfunctionswalk_treeand
walk_tree_recuraregenerators.Whencalled,theyreturnaniterator.They
dothisbecausetheyeachcontainayieldstatement.
Thesemethods/functionsarerecursive.Theycallthemselves.Sincetheyare
generators,theymustcallthemselvesinacontextthatusesaniterator,for
exampleinaforstatement.
1.7.4Modules
AmoduleisaPythonsourcecodefile.
Amodulecanbeimported.Whenimported,themoduleisevaluated,andamoduleobject
iscreated.Themoduleobjecthasattributes.Thefollowingattributesareofspecial
interest:
__doc__Thedocstringofthemodule.
__name__Thenameofthemodulewhenthemoduleisimported,butthe
string"__main__"whenthemoduleisexecuted.
Othernamesthatarecreated(bound)inthemodule.
Amodulecanberun.
Tomakeamodulebothimportableandrunable,usethefollowingidiom(attheendof
Page67
APythonBook
themodule):
defmain():
o
o
o
main()
WherePythonlooksformodules:
Seesys.path.
Standardplaces.
EnvironmentvariablePYTHONPATH.
Notesaboutmodulesandobjects:
Amoduleisanobject.
Amodule(object)canbeshared.
Aspecificmoduleisimportedonlyonceinasinglerun.Thismeansthatasingle
moduleobjectissharedbyallthemodulesthatimportit.
1.7.4.1Docstringsformodules
Adddocstringsasatriplequotedstringatornearthetopofthefile.Seeepydocfora
suggestedformat.
1.7.5Packages
Apackageisadirectoryonthefilesystemwhichcontainsafilenamed__init__.py.
The__init__.pyfile:
Whyisitthere?Itmakesmodulesinthedirectory"importable".
Can__init__.pybeempty?Yes.Or,justincludeacomment.
Whenisitevaluated?Itisevaluatedthefirsttimethatanapplicationimports
anythingfromthatdirectory/package.
Whatcanyoudowithit?Anycodethatshouldbeexecutedexactlyonceand
duringimport.Forexample:
Performinitializationneededbythepackage.
Makevariables,functions,classes,etcavailable.Forexample,whenthe
packageisimportedratherthanmodulesinthepackage.Youcanalsoexpose
objectsdefinedinmodulescontainedinthepackage.
Defineavariablenamed__all__tospecifythelistofnamesthatwillbe
importedbyfrommy_packageimport*.Forexample,ifthefollowingis
presentinmy_package/__init__.py:
Page68
APythonBook
__all__=['func1','func2',]
Then,frommy_packageimport*willimportfunc1andfunc2,but
notothernamesdefinedinmy_package.
Notethat__all__canbeusedatthemodulelevel,aswellasatthepackage
level.
Formoreinformation,seethesectiononpackagesinthePythontutorial:
http://docs.python.org/2/tutorial/modules.html#packages.
Guidanceandsuggestions:
"Flatisbetter"Usethe__init__.pyfiletopresenta"flat"viewoftheAPI
foryourcode.Enableyouruserstodoimportmypackageandthen
reference:
mypackage.item1
mypackage.item2
mypackage.item3
Etc.
Whereitem1,item2,etccomposetheAPIyouwantyouruserstouse,even
thoughtheimplementationoftheseitemsmaybeburieddeepinyourcode.
Usethe__init__.pymoduletopresenta"clean"API.Presentonlytheitems
thatyouintendyouruserstouse,andbydoingso,"hide"itemsyoudonotintend
themtouse.
1.8Classes
Classesmodelthebehaviorofobjectsinthe"real"world.Methodsimplementthe
behaviorsofthesetypesofobjects.Membervariableshold(current)state.Classesenable
ustoimplementnewdatatypesinPython.
Theclass:statementisusedtodefineaclass.Theclass:statementcreatesaclass
objectandbindsittoaname.
1.8.1Asimpleclass
In[104]:classA:
.....:pass
.....:
In[105]:a=A()
Todefineanewstyleclass(recommended),inheritfromobjectorfromanotherclass
thatdoes.Example:
In[21]:classA(object):
....:pass
....:
Page69
APythonBook
In[22]:
In[22]:a=A()
In[23]:a
Out[23]:<__main__.Aobjectat0x82fbfcc>
1.8.2Definingmethods
Amethodisafunctiondefinedinclassscopeandwithfirstparameterself:
In[106]:classB(object):
.....:defshow(self):
.....:print'hellofromB'
.....:
In[107]:b=B()
In[108]:b.show()
hellofromB
Amethodaswedescribeithereismoreproperlycalledaninstancemethod,inorderto
distinguishitfromclassmethodsandstaticmethods.
1.8.3Theconstructor
Theconstructorisamethodnamed__init__.
Exercise:Defineaclasswithamembervariablenameandashowmethod.Useprint
toshowthename.Solution:
.....:def__init__(self,name):
.....:self.name=name
.....:defshow(self):
.....:print'name:"%s"'%self.name
.....:
In[111]:a=A('dave')
In[112]:a.show()
name:"dave"
Notes:
Theselfvariableisexplicit.Itreferencesthecurrentobject,thatistheobject
whosemethodiscurrentlyexecuting.
Thespelling("self")isoptional,buteveryonespellsitthatway.
1.8.4Membervariables
DefiningmembervariablesMembervariablesarecreatedwithassignment.Example:
classA(object):
def__init__(self,name):
Page70
APythonBook
self.name=name
AsmallgotchaDothis:
....:def__init__(self,items=None):
....:ifitemsisNone:
....:self.items=[]
....:else:
....:self.items=items
Donotdothis:
In[29]:classB:
....:def__init__(self,items=[]):#wrong.listctor
evaluatedonlyonce.
....:self.items=items
Inthesecondexample,thedefstatementandthelistconstructorareevaluatedonly
once.Therefore,theitemmembervariableofallinstancesofclassB,willsharethesame
value,whichismostlikelynotwhatyouwant.
1.8.5Callingmethods
Usetheinstanceandthedotoperator.
Callingamethoddefinedinthesameclassorasuperclass:
FromoutsidetheclassUsetheinstance:
some_object.some_method()
an_array_of_of_objects[1].another_method()
FromwithinthesameclassUseself:
self.a_method()
Fromwithasubclasswhenthemethodisinthesuperclassandthereisa
methodwiththesamenameinthecurrentclassUsetheclass(name)oruse
super:
SomeSuperClass.__init__(self,arg1,arg2)
super(CurrentClass,
self).__init__(arg1,arg2)
CallingamethoddefinedinaspecificsuperclassUsetheclass(name).
1.8.6Addinginheritance
ReferencingsuperclassesUsethebuiltinsuperortheexplicitnameofthe
superclass.Useofsuperispreferred.Forexample:
In[39]:classB(A):
Page71
APythonBook
....:def__init__(self,name,size):
....:super(B,self).__init__(name)
....:#A.__init__(self,name)#anolderalternative
form
....:self.size=size
Theuseofsuper()maysolveproblemssearchingforthebaseclasswhenusing
multipleinheritance.Abettersolutionistonotusemultipleinheritance.
Youcanalsousemultipleinheritance.But,itcancauseconfusion.Forexample,inthe
following,classCinheritsfrombothAandB:
classC(A,B):
...
PythonsearchessuperclassesMRO(methodresolutionorder).Ifonlysingleinheritance
isinvolved,thereislittleconfusion.Ifmultipleinheritanceisbeingused,thesearchorder
ofsuperclassescangetcomplexseehere:
http://www.python.org/download/releases/2.3/mro
Formoreinformationoninheritance,seethetutorialinthestandardPython
documentationset:9.5Inheritanceand9.5.1MultipleInheritance.
Watchoutforproblemswithinheritingfrommultipleclassesthathaveacommonbase
class.
1.8.7Classvariables
Alsocalledstaticdata.
Aclassvariableissharedbyinstancesoftheclass.
Defineatclasslevelwithassignment.Example:
classA(object):
size=5
defget_size(self):
returnA.size
Referencewithclassname.variable.
Caution:self.variable=xcreatesanewmembervariable.
1.8.8Classmethodsandstaticmethods
Instance(plain)methods:
Aninstancemethodreceivestheinstanceasitsfirstargument.
Classmethods:
Aclassmethodreceivestheclassasitsfirstargument.
Defineclassmethodswithbuiltinfunctionclassmethod()orwithdecorator
Page72
APythonBook
@classmethod.
Seethedescriptionofclassmethod()builtinfunctionat"Builtin
Functions":http://docs.python.org/2/library/functions.html#classmethod
Staticmethods:
Astaticmethodreceivesneithertheinstancenortheclassasitsfirstargument.
Definestaticmethodswithbuiltinfunctionstaticmethod()orwith
decorator@staticmethod.
Seethedescriptionofstaticmethod()builtinfunctionat"Builtin
Functions":http://docs.python.org/2/library/functions.html#staticmethod
Notesondecorators:
Adecoratoroftheform@afuncisthesameasm=afunc(m).So,this:
@afunc
defm(self):pass
isthesameas:
defm(self):pass
m=afunc(m)
Youcanusedecorators@classmethodand@staticmethod(insteadofthe
classmethod()andstaticmethod()builtinfunctions)todeclareclass
methodsandstaticmethods.
Example:
classB(object):
Count=0
defdup_string(x):
s1='%s%s'%(x,x,)
returns1
dup_string=staticmethod(dup_string)
@classmethod
defshow_count(cls,msg):
print'%s%d'%(msg,cls.Count,)
deftest():
printB.dup_string('abcd')
B.show_count('hereisthecount:')
Analternativewaytoimplement"staticmethods"Usea"plain",modulelevel
function.Forexample:
In[1]:definc_count():
...:A.count+=1
...:
In[2]:
Page73
APythonBook
In[2]:defdec_count():
...:A.count=1
...:
In[3]:
In[3]:classA:
...:count=0
...:defget_count(self):
...:returnA.count
...:
In[4]:
In[4]:a=A()
In[5]:a.get_count()
Out[5]:0
In[6]:
In[6]:
In[6]:inc_count()
In[7]:inc_count()
In[8]:a.get_count()
Out[8]:2
In[9]:
In[9]:b=A()
In[10]:b.get_count()
Out[10]:2
1.8.9Properties
Thepropertybuiltinfunctionenablesustowriteclassesinawaythatdoesnotrequirea
useroftheclasstousegettersandsetters.Example:
classTestProperty(object):
def__init__(self,description):
self._description=description
def_set_description(self,description):
print'settingdescription'
def_get_description(self):
print'gettingdescription'
returnself._description
description=property(_get_description,_set_description)
Thepropertybuiltinfunctionisalsoadecorator.So,thefollowingisequivalentto
theaboveexample:
classTestProperty(object):
def__init__(self,description):
@property
defdescription(self):
print'gettingdescription'
returnself._description
Page74
APythonBook
@description.setter
defdescription(self,description):
print'settingdescription'
Notes:
Wemarktheinstancevariableasprivatebyprefixingitwithandunderscore.
Thenameoftheinstancevariableandthenameofthepropertymustbedifferent.
Iftheyarenot,wegetrecursionandanerror.
Formoreinformationonproperties,seeBuiltinFunctionsproperties
http://docs.python.org/2/library/functions.html#property
1.8.10Interfaces
InPython,toimplementaninterfaceistoimplementamethodwithaspecificnameanda
specificarguments.
"Ducktyping"Ifitwalkslikeaduckandquackslikeaduck...
Onewaytodefinean"interface"istodefineaclasscontainingmethodsthathavea
headerandadocstringbutnoimplementation.
Additionalnotesoninterfaces:
Interfacesarenotenforced.
Aclassdoesnothavetoimplementallofaninterface.
1.8.11Newstyleclasses
Anewstyleclassisonethatsubclassesobjectoraclassthatsubclassesobject(that
is,anothernewstyleclass).
YoucansubclassPython'sbuiltindatatypes.
Asimpleexamplethefollowingclassextendsthelistdatatype:
classC(list):
defget_len(self):
returnlen(self)
c=C((11,22,33))
c.get_len()
c=C((11,22,33,44,55,66,77,88))
printc.get_len()
#Prints"8".
Aslightlymorecomplexexamplethefollowingclassextendsthedictionary
Page75
APythonBook
datatype:
classD(dict):
def__init__(self,data=None,name='no_name'):
ifdataisNone:
data={}
dict.__init__(self,data)
self.name=name
defget_len(self):
returnlen(self)
defget_keys(self):
content=[]
forkeyinself:
content.append(key)
contentstr=','.join(content)
returncontentstr
defget_name(self):
returnself.name
deftest():
d=D({'aa':111,'bb':222,'cc':333})
#Prints"3"
printd.get_len()
#Prints"'aa,cc,bb'"
printd.get_keys()
#Prints"no_name"
printd.get_name()
Somethingstorememberaboutnewstyleclasses:
Inordertobenewstyle,aclassmustinherit(directlyorindirectly)from
object.Notethatifyouinheritfromabuiltintype,yougetthisautomatically.
Newstyleclassesunifytypesandclasses.
Youcansubclass(builtin)typessuchasdict,str,list,file,etc.
Thebuiltintypesnowprovidefactoryfunctions:dict(),str(),int(),
file(),etc.
ThebuiltintypesareintrospectableUsex.__class__,
dir(x.__class__),isinstance(x,list),etc.
Newstyleclassesgiveyoupropertiesanddescriptors.
Newstyleclassesenableyoutodefinestaticmethods.Actually,allclassesenable
youtodothis.
Anewstyleclassisauserdefinedtype.Foraninstanceofanewstyleclassx,
type(x)isthesameasx.__class__.
Formoreonnewstyleclasses,see:http://www.python.org/doc/newstyle/
Exercises:
Writeaclassandasubclassofthisclass.
Givethesuperclassonemembervariable,aname,whichcanbeenteredwhen
Page76
APythonBook
aninstanceisconstructed.
Givethesubclassonemembervariable,adescription;thesubclassconstructor
shouldallowentryofbothnameanddescription.
Putashow()methodinthesuperclassandoverridetheshow()methodin
thesubclass.
Solution:
classA(object):
def__init__(self,name):
self.name=name
defshow(self):
print'name:%s'%(self.name,)
classB(A):
def__init__(self,name,desc):
A.__init__(self,name)
self.desc=desc
defshow(self):
A.show(self)
print'desc:%s'%(self.desc,)
1.8.12Docstringsforclasses
Adddocstringsasa(triplequoted)stringbeginningwiththefirstlineofaclass.See
epydocforasuggestedformat.
1.8.13Privatemembers
Addanleadingunderscoretoamembername(methodordatavariable)to"suggest"that
thememberisprivate.
1.9SpecialTasks
1.9.1Debuggingtools
pdbThePythondebugger:
Startthedebuggerbyrunninganexpression:
pdb.run('expression')
Example:
importpdb
pdb.run('main()')
Startupthedebuggerataspecificlocationwiththefollowing:
Page77
APythonBook
importpdb;pdb.set_trace()
Example:
importpdb
pdb.set_trace()
main()
Gethelpfromwithinthedebugger.Forexample:
(Pdb)help
(Pdb)helpnext
CanalsoembedIPythonintoyourcode.See
http://ipython.scipy.org/doc/manual/manual.html.
ipdbAlsoconsiderusingipdb(andIPython).Theipdbdebuggerinteractive
prompthassomeadditionalfeatures,forexample,itdoestabnamecompletion.
Inspecting:
importinspect
Seehttp://docs.python.org/lib/moduleinspect.html.
Don'tforgettotrydir(obj)andtype(obj)andhelp(obj),first.
Miscellaneoustools:
id(obj)
globals()andlocals().
dir(obj)Returnsinterestingnames,butlistisnotnecessarilycomplete.
obj.__class__
cls.__bases__
obj.__class__.__bases__
obj.__doc__.Butusually,help(obj)isbetter.Itproducesthedocstring.
Customizetherepresentationofyourclass.Definethefollowingmethodsinyour
class:
__repr__()Calledby(1)repr(),(2)interactiveinterpreterwhen
representationisneeded.
__str__()Calledby(1)str(),(2)stringformatting.
pdbisimplementedwiththecmdmoduleinthePythonstandardlibrary.Youcan
implementsimilarcommandlineinterfacesbyusingcmd.See:cmdSupportfor
lineorientedcommandinterpretershttp://docs.python.org/lib/modulecmd.html.
1.9.2Fileinputandoutput
Createafileobject.Useopen().
Thisexamplereadsandprintseachlineofafile:
Page78
APythonBook
deftest():
f=file('tmp.py','r')
forlineinf:
print'line:',line.rstrip()
f.close()
test()
Notes:
Atextfileisaniterable.Ititeratesoverthelinesinafile.Thefollowingisa
commonidiom:
infile=file(filename,'r')
forlineininfile:
process_a_line(line)
infile.close()
string.rstrip()stripsnewlineandotherwhitespacefromtherightsideof
eachline.Tostripnewlinesonly,butnototherwhitespace,tryrstrip('\n').
Otherwaysofreadingfromafile/streamobject:my_file.read(),
my_file.readline(),my_file.readlines(),
Thisexamplewriteslinesoftexttoafile:
deftest():
f=file('tmp.txt','w')
forchin'abcdefg':
f.write(ch*10)
f.write('\n')
f.close()
test()
Notes:
Thewritemethod,unliketheprintstatement,doesnotautomaticallyadd
newlinecharacters.
Mustclosefileinordertoflushoutput.Or,usemy_file.flush().
And,don'tforgetthewith:statement.Itmakesclosingfilesautomatic.Thefollowing
exampleconvertsallthevowelsinaninputfiletouppercaseandwritestheconverted
linestoanoutputfile:
importstring
defshow_file(infilename,outfilename):
tran_table=string.maketrans('aeiou','AEIOU')
withopen(infilename,'r')asinfile,open(outfilename,'w')as
outfile:
forlineininfile:
line=line.rstrip()
outfile.write('%s\n'%line.translate(tran_table))
Page79
APythonBook
1.9.3Unittests
Formoredocumentationontheunittestframework,seeunittestUnittesting
frameworkhttp://docs.python.org/2/library/unittest.html#moduleunittest
ForhelpandmoreinformationdothefollowingatthePythoninteractiveprompt:
>>>importunittest
>>>help(unittest)
And,youcanreadthesource:Lib/unittest.pyinthePythonstandardlibrary.
1.9.3.1Asimpleexample
Hereisaverysimpleexample.Youcanfindmoreinformationaboutthisprimitiveway
ofstructuringunittestsinthelibrarydocumentationfortheunittestmoduleBasic
examplehttp://docs.python.org/lib/minimalexample.html
importunittest
classUnitTests02(unittest.TestCase):
deftestFoo(self):
self.failUnless(False)
deftestBar01(self):
deftestBar02(self):
defmain():
unittest.main()
main()
Notes:
Thecalltounittest.main()runsalltestsinalltestfixturesinthemodule.It
actuallycreatesaninstanceofclassTestPrograminmodule
Lib/unittest.py,whichautomaticallyrunstests.
Testfixturesareclassesthatinheritfromunittest.TestCase.
Withinatestfixture(aclass),thetestsareanymethodswhosenamesbeginwith
theprefix"test".
Inanytest,wecheckforsuccessorfailurewithinheritedmethodssuchas
failIf(),failUnless(),assertNotEqual(),etc.Formoreonthese
Page80
APythonBook
methods,seethelibrarydocumentationfortheunittestmoduleTestCase
Objectshttp://docs.python.org/lib/testcaseobjects.html.
Ifyouwanttochange(1)thetestmethodprefixor(2)thefunctionusedtosort
(theorderof)executionoftestswithinatestfixture,thenyoucancreateyourown
instanceofclassunittest.TestLoaderandcustomizeit.Forexample:
defmain():
my_test_loader=unittest.TestLoader()
my_test_loader.testMethodPrefix='check'
my_test_loader.sortTestMethodsUsing=my_cmp_func
unittest.main(testLoader=my_test_loader)
main()
But,seethenotesinsectionAdditionalunittestfeaturesforinstructionsona
(possibly)betterwaytodothis.
1.9.3.2Unittestsuites
Hereisanother,notquitesosimple,example:
#!/usr/bin/envpython
importsys,popen2
importgetopt
importunittest
classGenTest(unittest.TestCase):
deftest_1_generate(self):
cmd='python../generateDS.pyfoout2sup.pysout2sub.py
people.xsd'
outfile,infile=popen2.popen2(cmd)
result=outfile.read()
outfile.close()
infile.close()
self.failUnless(len(result)==0)
deftest_2_compare_superclasses(self):
cmd='diffout1sup.pyout2sup.py'
outfile.close()
infile.close()
#print'len(result):',len(result)
#Ignorethedifferinglinescontainingthedate/time.
#self.failUnless(len(result)<130and
result.find('Generated')>1)
Page81
APythonBook
self.failUnless(check_result(result))
deftest_3_compare_subclasses(self):
cmd='diffout1sub.pyout2sub.py'
outfile.close()
infile.close()
#Ignorethedifferinglinescontainingthedate/time.
#self.failUnless(len(result)<130and
result.find('Generated')>1)
self.failUnless(check_result(result))
defcheck_result(result):
flag1=0
flag2=0
lines=result.split('\n')
len1=len(lines)
iflen1<=5:
flag1=1
s1='\n'.join(lines[:4])
ifs1.find('Generated')>1:
flag2=1
returnflag1andflag2
#Makethetestsuite.
defsuite():
#Thefollowingisobsolete.SeeLib/unittest.py.
#returnunittest.makeSuite(GenTest)
loader=unittest.TestLoader()
#oralternatively
#loader=unittest.defaultTestLoader
testsuite=loader.loadTestsFromTestCase(GenTest)
returntestsuite
#Makethetestsuiteandrunthetests.
deftest():
testsuite=suite()
runner=unittest.TextTestRunner(sys.stdout,verbosity=2)
runner.run(testsuite)
USAGE_TEXT="""
Usage:
pythontest.py[options]
Options:
h,helpDisplaythishelpmessage.
Example:
pythontest.py
Page82
APythonBook
"""
defusage():
printUSAGE_TEXT
sys.exit(1)
defmain():
args=sys.argv[1:]
try:
opts,args=getopt.getopt(args,'h',['help'])
except:
usage()
relink=1
foropt,valinopts:
ifoptin('h','help'):
usage()
iflen(args)!=0:
usage()
test()
main()
#importpdb
#pdb.run('main()')
Notes:
GenTestisourtestsuiteclass.Itinheritsfromunittest.TestCase.
EachmethodinGenTestwhosenamebeginswith"test"willberunasatest.
Thetestsareruninalphabeticorderbymethodname.
DefaultsinclassTestLoaderforthetestnameprefixandsortcomparison
functioncanbeoverridden.See5.3.8TestLoaderObjects
http://docs.python.org/lib/testloaderobjects.html.
AtestcaseclassmayalsoimplementmethodsnamedsetUp()and
tearDown()toberunbeforeandaftertests.See5.3.5TestCaseObjects
http://docs.python.org/lib/testcaseobjects.html.Actually,thefirsttestmethodin
ourexampleshould,perhaps,beasetUp()method.
Thetestsusecallssuchasself.failUnless()toreporterrors.Theseare
inheritedfromclassTestCase.See5.3.5TestCaseObjects
http://docs.python.org/lib/testcaseobjects.html.
Functionsuite()createsaninstanceofthetestsuite.
Functiontest()runsthetests.
1.9.3.3Additionalunittestfeatures
And,thefollowingexampleshowsseveraladditionalfeatures.Seethenotesthatfollow
Page83
APythonBook
thecode:
importunittest
deftestFoo(self):
defcheckBar01(self):
#Note1
defsetUp(self):
print'settingupUnitTests01'
deftearDown(self):
print'tearingdownUnitTests01'
deftestBar01(self):
print'testingtestBar01'
deftestBar02(self):
print'testingtestBar02'
deffunction_test_1():
name='mona'
assertnotname.startswith('mo')
defcompare_names(name1,name2):
ifname1<name2:
return1
elifname1>name2:
return1
else:
return0
defmake_suite():
suite=unittest.TestSuite()
#Note2
suite.addTest(unittest.makeSuite(UnitTests01,
sortUsing=compare_names))
#Note3
suite.addTest(unittest.makeSuite(UnitTests02,prefix='check'))
#Note4
suite.addTest(unittest.FunctionTestCase(function_test_1))
returnsuite
defmain():
suite=make_suite()
runner=unittest.TextTestRunner()
runner.run(suite)
Page84
APythonBook
main()
Notes:
1. Ifyourunthiscode,youwillnoticethatthesetUpandtearDownmethodsin
classUnitTests01arerunbeforeandaftereachtestinthatclass.
2. Wecancontroltheorderinwhichtestsarerunbypassingacomparefunctionto
themakeSuitefunction.Thedefaultisthecmpbuiltinfunction.
3. Wecancontrolwhichmethodsinatestfixtureareselectedtoberunbypassing
theoptionalargumentprefixtothemakeSuitefunction.
4. Ifwehaveanexistingfunctionthatwewantto"wrap"andrunasaunittest,we
cancreateatestcasefromafunctionwiththeFunctionTestCasefunction.If
wedothat,noticethatweusetheassertstatementtotestandpossiblycause
failure.
1.9.3.4GuidanceonUnitTesting
Whyshouldweuseunittests?Manyreasons,including:
Withoutunittests,cornercasesmaynotbechecked.Thisisespeciallyimportant,
sincePythondoesrelativelylittlecompiletimeerrorchecking.
Unittestsfacilitateafrequentandshortdesignandimplementandrelease
developmentcycle.SeeONLamp.comExtremePython
http://www.onlamp.com/pub/a/python/2001/03/28/pythonnews.htmlandWhatis
XPhttp://www.xprogramming.com/what_is_xp.htm.
Designingthetestsbeforewritingthecodeis"agoodidea".
Additionalnotes:
Inatestclass,instancemethodssetUpandtearDownarerunautomatically
beforeeachandaftereachindividualtest.
Inatestclass,classmethodssetUpClassandtearDownClassarerun
automaticallyoncebeforeandafterallthetestsinaclass.
ModulelevelfunctionssetUpModuleandtearDownModulearerunbefore
andafteranytestsinamodule.
Insomecasesyoucanalsoruntestsdirectlyfromthecommandline.Dothe
followingforhelp:
$pythonmunittesthelp
1.9.4doctest
Forsimpletestharnesses,considerusingdoctest.Withdoctestyoucan(1)runa
testatthePythoninteractiveprompt,then(2)copyandpastethattestintoadocstringin
yourmodule,andthen(3)runthetestsautomaticallyfromwithinyourmoduleunder
Page85
APythonBook
doctest.
ThereareexamplesandexplanationinthestandardPythondocumentation:5.2doctest
TestinteractivePythonexampleshttp://docs.python.org/lib/moduledoctest.html.
Asimplewaytousedoctestinyourmodule:
1. RunseveraltestsinthePythoninteractiveinterpreter.Notethatbecause
doctestlooksfortheinterpreter's">>>"prompt,youmustusethestandard
interpreter,andnot,forexample,IPython.Also,makesurethatyouincludealine
withthe">>>"promptaftereachsetofresults;thisenablesdoctestto
determinetheextentofthetestresults.
2. Usecopyandpaste,toinsertthetestsandtheirresultsfromyourinteractive
sessionintothedocstrings.
3. Addthefollowingcodeatthebottomofyourmodule:
def_test():
importdoctest
doctest.testmod()
if__name__=="__main__":
_test()
Hereisanexample:
deff(n):
"""
Printsomethingfunny.
>>>f(1)
10
>>>f(2)
10
>>>f(3)
0
"""
ifn==1:
return10
elifn==2:
return10
else:
return0
deftest():
importdoctest,test_doctest
doctest.testmod(test_doctest)
test()
And,hereistheoutputfromrunningtheabovetestwiththevflag:
Page86
APythonBook
$pythontest_doctest.pyv
Runningtest_doctest.__doc__
0of0examplesfailedintest_doctest.__doc__
Runningtest_doctest.f.__doc__
Trying:f(1)
Expecting:10
ok
Trying:f(2)
Expecting:10
ok
Trying:f(3)
Expecting:0
ok
0of3examplesfailedintest_doctest.f.__doc__
Runningtest_doctest.test.__doc__
0of0examplesfailedintest_doctest.test.__doc__
2itemshadnotests:
test_doctest
test_doctest.test
1itemspassedalltests:
3testsintest_doctest.f
3testsin3items.
3passedand0failed.
Testpassed.
1.9.5ThePythondatabaseAPI
PythondatabaseAPIdefinesastandardinterfaceforaccesstoarelationaldatabase.
InordertousethisAPIyoumustinstallthedatabaseadapter(interfacemodule)foryour
particulardatabase,e.g.PostgreSQL,MySQL,Oracle,etc.
YoucanlearnmoreaboutthePythonDBAPIhere:
Thefollowingsimpleexampleusessqlite3http://docs.python.org/2/library/sqlite3.html
"""
Createarelationaldatabaseandatableinit.
Addsomerecords.
Readanddisplaytherecords.
"""
importsys
importsqlite3
defcreate_table(db_name):
con=sqlite3.connect(db_name)
cursor=con.cursor()
cursor.execute('''CREATETABLEplants
Page87
APythonBook
(nametext,desctext,catint)''')
cursor.execute(
'''INSERTINTOplantsVALUES('tomato','redandjuicy',
1)''')
cursor.execute(
'''INSERTINTOplantsVALUES('pepper','greenandcrunchy',
2)''')
cursor.execute('''INSERTINTOplantsVALUES('pepper','purple',
2)''')
con.commit()
con.close()
defretrieve(db_name):
con=sqlite3.connect(db_name)
cursor=con.cursor()
cursor.execute('''select*fromplants''')
rows=cursor.fetchall()
printrows
print''*40
cursor.execute('''select*fromplants''')
forrowincursor:
printrow
con.close()
deftest():
args=sys.argv[1:]
iflen(args)!=1:
sys.stderr.write('\nusage:test_db.py<db_name>\n\n')
sys.exit(1)
db_name=args[0]
create_table(db_name)
retrieve(db_name)
test()
1.9.6InstallingPythonpackages
Simple:
$pythonsetup.pybuild
$pythonsetup.pyinstall#asroot
Morecomplex:
LookforaREADMEorINSTALLfileattherootofthepackage.
Typethefollowingforhelp:
$pythonsetup.pycmdhelp
$pythonsetup.pyhelpcommands
$pythonsetup.pyhelp[cmd1cmd2...]
And,forevenmoredetails,seeInstallingPythonModules
Page88
APythonBook
http://docs.python.org/inst/inst.html
pipisbecomingpopularforinstallingandmanagingPythonpackages.See:
https://pypi.python.org/pypi/pip
Also,considerusingvirtualenv,especiallyifyoususpectorworrythatinstalling
somenewpackagewillalterthebehaviorofapackagecurrentlyinstalledonyour
machine.See:https://pypi.python.org/pypi/virtualenv.virtualenvcreatesadirectory
andsetsupaPythonenvironmentintowhichyoucaninstallandusePythonpackages
withoutchangingyourusualPythoninstallation.
1.10MorePythonFeaturesandExercises
[Astimepermits,explainmorefeaturesanddomoreexercisesasrequestedbyclass
members.]
ThankstoDavidGoodgerforthefollowinglistorreferences.His"CodeLikea
Pythonista:IdiomaticPython"
(http://python.net/~goodger/projects/pycon/2007/idiomatic/)isworthacarefulreading:
"PythonObjects",FredrikLundh,http://www.effbot.org/zone/pythonobjects.htm
"HowtothinklikeaPythonista",MarkHammond,
http://python.net/crew/mwh/hacks/objectthink.html
"Pythonmain()functions",GuidovanRossum,
http://www.artima.com/weblogs/viewpost.jsp?thread=4829
"PythonIdiomsandEfficiency",http://jaynes.colorado.edu/PythonIdioms.html
"Pythontrack:pythonidioms",
http://www.cs.caltech.edu/courses/cs11/material/python/misc/python_idioms.html
"BePythonic",ShalabhChaturvedi,http://shalabh.infogami.com/Be_Pythonic2
"PythonIsNotJava",PhillipJ.Eby,
http://dirtsimple.org/2004/12/pythonisnotjava.html
"WhatisPythonic?",MartijnFaassen,
http://faassen.ntree.net/blog/view/weblog/2005/08/06/0
"SortingMiniHOWTO",AndrewDalke,
http://wiki.python.org/moin/HowTo/Sorting
"PythonIdioms",http://www.gungfu.de/facts/wiki/Main/PythonIdioms
"PythonFAQs",http://www.python.org/doc/faq/
Page89
APythonBook
2Part2AdvancedPython
2.1IntroductionPython201(Slightly)AdvancedPythonTopics
Thisdocumentisintendedasnotesforacourseon(slightly)advancedPythontopics.
2.2RegularExpressions
Formorehelponregularexpressions,see:
reRegularexpressionoperationshttp://docs.python.org/library/re.html
RegularExpressionHOWTOhttp://docs.python.org/howto/regex.html
2.2.1Definingregularexpressions
Aregularexpressionpatternisasequenceofcharactersthatwillmatchsequencesof
charactersinatarget.
Thepatternsorregularexpressionscanbedefinedasfollows:
Literalcharactersmustmatchexactly.Forexample,"a"matches"a".
Concatenatedpatternsmatchconcatenatedtargets.Forexample,"ab"("a"
followedby"b")matches"ab".
Alternatepatterns(separatedbyaverticalbar)matcheitherofthealternative
patterns.Forexample,"(aaa)|(bbb)"willmatcheither"aaa"or"bbb".
Repeatingandoptionalitems:
"abc*"matches"ab"followedbyzeroormoreoccurancesof"c",forexample,
"ab","abc","abcc",etc.
"abc+"matches"ab"followedbyoneormoreoccurancesof"c",forexample,
"abc","abcc",etc,butnot"ab".
"abc?"matches"ab"followedbyzerooroneoccurancesof"c",forexample,
"ab"or"abc".
SetsofcharactersCharactersandsequencesofcharactersinsquarebrackets
formaset;asetmatchesanycharacterinthesetorrange.Forexample,"[abc]"
matches"a"or"b"or"c".And,forexample,"[_az09]"matchesanunderscore
oranylowercaseletteroranydigit.
GroupsParenthesesindicateagroupwithapattern.Forexample,"ab(cd)*ef"is
apatternthatmatches"ab"followedbyanynumberofoccurancesof"cd"
followedby"ef",forexample,"abef","abcdef","abcdcdef",etc.
Therearespecialnamesforsomesetsofcharacters,forexample"\d"(anydigit),
Page90
APythonBook
"\w"(anyalphanumericcharacter),"\W"(anynonalphanumericcharacter),etc.
Moremoreinformation,seePythonLibraryReference:RegularExpression
Syntaxhttp://docs.python.org/library/re.html#regularexpressionsyntax
Becauseoftheuseofbackslashesinpatterns,youareusuallybetteroffdefiningregular
expressionswithrawstrings,e.g.r"abc".
2.2.2Compilingregularexpressions
Whenaregularexpressionistobeusedmorethanonce,youshouldconsidercompiling
it.Forexample:
importsys,re
pat=re.compile('aa[bc]*dd')
while1:
line=raw_input('Enteraline("q"toquit):')
ifline=='q':
break
ifpat.search(line):
print'matched:',line
else:
print'nomatch:',line
Comments:
Weimportmodulereinordertouseregularexpresions.
re.compile()compilesaregularexpressionsothatwecanreusethe
compiledregularexpressionwithoutcompilingitrepeatedly.
2.2.3Usingregularexpressions
Usematch()tomatchatthebeginningofastring(ornotatall).
Usesearch()tosearchastringandmatchthefirststringfromtheleft.
Herearesomeexamples:
>>>importre
>>>pat=re.compile('aa[09]*bb')
>>>x=pat.match('aa1234bbccddee')
>>>x
<_sre.SRE_Matchobjectat0x401e9608>
>>>x=pat.match('xxxxaa1234bbccddee')
>>>x
>>>type(x)
<type'NoneType'>
>>>x=pat.search('xxxxaa1234bbccddee')
>>>x
Page91
APythonBook
Notes:
Whenamatchorsearchissuccessful,itreturnsamatchobject.Whenitfails,it
returnsNone.
Youcanalsocallthecorrespondingfunctionsmatchandsearchintheremodule,
e.g.:
>>>x=re.search(pat,'xxxxaa1234bbccddee')
>>>x
Foralistoffunctionsintheremodule,seeModuleContents
http://docs.python.org/library/re.html#modulecontents.
2.2.4Usingmatchobjectstoextractavalue
Matchobjectsenableyoutoextractmatchedsubstringsafterperformingamatch.A
matchobjectisreturnedbysuccessfulmatch.Thepartofthetargetavailableinthematch
objectistheportionmatchedbygroupsinthepattern,thatistheportionofthepattern
insideparentheses.Forexample:
In[69]:mo=re.search(r'height:(\d*)width:(\d*)','height:123
width:456')
In[70]:mo.groups()
Out[70]:('123','456')
Hereisanotherexample:
importsys,re
Targets=[
'Thereare<<25>>sparrows.',
'Isee<<15>>finches.',
'Thereisnothinghere.',
]
deftest():
pat=re.compile('<<([09]*)>>')
forlineinTargets:
mo=pat.search(line)
ifmo:
value=mo.group(1)
print'value:%s'%value
else:
print'nomatch'
test()
Whenweruntheabove,itprintsoutthefollowing:
Page92
APythonBook
value:25
value:15
nomatch
Explanation:
Intheregularexpression,putparenthesesaroundtheportionoftheregular
expressionthatwillmatchwhatyouwanttoextract.Eachpairofparentheses
marksoffagroup.
Afterthesearch,checktodetermineiftherewasasuccessfulmatchbychecking
foramatchingobject."pat.search(line)"returnsNoneifthesearchfails.
Ifyouspecifymorethanonegroupinyourregularexpression(morethatonepair
ofparentheses),thenyoucanuse"value=mo.group(N)"toextractthevalue
matchedbytheNthgroupfromthematchingobject."value=mo.group(1)"
returnsthefirstextractedvalue;"value=mo.group(2)"returnsthesecond;etc.An
argumentof0returnsthestringmatchedbytheentireregularexpression.
Inaddition,youcan:
Use"values=mo.groups()"togetatuplecontainingthestringsmatchedbyall
groups.
Use"mo.expand()"tointerpolatethegroupvaluesintoastring.Forexample,
"mo.expand(r'value1:\1value2:\2')"insertsthevaluesofthefirstandsecond
groupintoastring.Ifthefirstgroupmatched"aaa"andthesecondmatched
"bbb",thenthisexamplewouldproduce"value1:aaavalue2:bbb".Forexample:
In[76]:mo=re.search(r'h:(\d*)w:(\d*)','h:123
w:456')
In[77]:mo.expand(r'Height:\1Width:\2')
Out[77]:'Height:123Width:456'
2.2.5Extractingmultipleitems
Youcanextractmultipleitemswithasinglesearch.Hereisanexample:
importsys,re
pat=re.compile('aa([09]*)bb([09]*)cc')
while1:
ifline=='q':
break
mo=pat.search(line)
ifmo:
value1,value2=mo.group(1,2)
print'value1:%svalue2:%s'%(value1,value2)
else:
print'nomatch'
Page93
APythonBook
Comments:
Usemultipleparenthesizedsubstringsintheregularexpressiontoindicatethe
portions(groups)tobeextracted.
"mo.group(1,2)"returnsthevaluesofthefirstandsecondgroupinthestring
matched.
Wecouldalsohaveused"mo.groups()"toobtainatuplethatcontainsboth
values.
Yetanotheralternativewouldhavebeentousethefollowing:print
mo.expand(r'value1:\1value2:\2').
2.2.6Replacingmultipleitems
Asimplewaytoperformmultiplereplacementsusingaregularexpressionistousethe
re.subn()function.Hereisanexample:
In[81]:re.subn(r'\d+','***','thereare203birdssittingin2
trees')
Out[81]:('thereare***birdssittingin***trees',2)
Formorecomplexreplacements,useafunctioninsteadofaconstantreplacementstring:
importre
defrepl_func(mo):
s1=mo.group(1)
s2='*'*len(s1)
returns2
deftest():
pat=r'(\d+)'
in_str='thereare2034birdsin21trees'
out_str,count=re.subn(pat,repl_func,in_str)
print'in:"%s"'%in_str
print'out:"%s"'%out_str
print'count:%d'%count
test()
Andwhenweruntheabove,itproduces:
in:"thereare2034birdsin21trees"
out:"thereare****birdsin**trees"
count:2
Notes:
Thereplacementfunctionreceivesoneargument,amatchobject.
There.subn()functionreturnsatuplecontainingtwovalues:(1)thestring
afterreplacementsand(2)thenumberofreplacementsperformed.
Page94
APythonBook
HereisanevenmorecomplexexampleYoucanlocatesubstrings(slices)ofamatch
andreplacethem:
importsys,re
pat=re.compile('aa([09]*)bb([09]*)cc')
while1:
ifline=='q':
break
mo=pat.search(line)
ifmo:
value1,value2=mo.group(1,2)
start1=mo.start(1)
end1=mo.end(1)
start2=mo.start(2)
end2=mo.end(2)
print'value1:%sstart1:%dend1:%d'%(value1,start1,
end1)
print'value2:%sstart2:%dend2:%d'%(value2,start2,
end2)
repl1=raw_input('Enterreplacement#1:')
repl2=raw_input('Enterreplacement#2:')
newline=(line[:start1]+repl1+line[end1:start2]+
repl2+line[end2:])
print'newline:%s'%newline
else:
print'nomatch'
Explanation:
Alternatively,use"mo.span(1)"insteadof"mo.start(1)"and"mo.end(1)"inorder
togetthestartandendofasubmatchinasingleoperation."mo.span(1)"returnsa
tuple:(start,end).
Puttogetheranewstringwithstringconcatenationfrompiecesoftheoriginal
stringandreplacementvalues.Youcanusestringslicestogetthesubstringsof
theoriginalstring.Inourcase,thefollowinggetsthestartofthestring,addsthe
firstreplacement,addsthemiddleoftheoriginalstring,addsthesecond
replacement,andfinally,addsthelastpartoftheoriginalstring:
newline=line[:start1]+repl1+line[end1:start2]+
repl2+line[end2:]
Youcanalsousethesubfunctionormethodtodosubstitutions.Hereisanexample:
importsys,re
pat=re.compile('[09]+')
print'Replacingdecimaldigits.'
Page95
APythonBook
while1:
target=raw_input('Enteratargetline("q"toquit):')
iftarget=='q':
break
repl=raw_input('Enterareplacement:')
result=pat.sub(repl,target)
print'result:%s'%result
Hereisanotherexampleoftheuseofafunctiontoinsertcalculatedreplacements.
importsys,re,string
pat=re.compile('[am]+')
defreplacer(mo):
returnstring.upper(mo.group(0))
print'Uppercasingam.'
while1:
iftarget=='q':
break
result=pat.sub(replacer,target)
Notes:
Ifthereplacementargumenttosubisafunction,thatfunctionmusttakeone
argument,amatchobject,andmustreturnthemodified(orreplacement)value.
Thematchedsubstringwillbereplacedbythevaluereturnedbythisfunction.
Inourcase,thefunctionreplacerconvertsthematchedvaluetouppercase.
Thisisalsoaconvenientuseforalambdainsteadofanamedfunction,forexample:
importsys,re,string
pat=re.compile('[am]+')
print'Uppercasingam.'
while1:
iftarget=='q':
break
result=pat.sub(
lambdamo:string.upper(mo.group(0)),
target)
2.3IteratorObjects
Note1:YouwillneedasufficientlyrecentversionofPythoninordertouseiteratorsand
generators.IbelievethattheywereintroducedinPython2.2.
Page96
APythonBook
Note2:TheiteratorprotocolhaschangedslightlyinPythonversion3.0.
Goalsforthissection:
Learnhowtoimplementageneratorfunction,thatis,afunctionwhich,when
called,returnsaniterator.
Learnhowtoimplementaclasscontainingageneratormethod,thatis,amethod
which,whencalled,returnsaniterator.
Learntheiteratorprotocol,specificallywhatmethodsaniteratormustsupportand
whatthosemethodsmustdo.
Learnhowtoimplementaniteratorclass,thatis,aclasswhoseinstancesare
iteratorobjects.
Learnhowtoimplementrecursiveiteratorgenerators,thatis,aniteratorgenerator
whichrecursivelyproducesiteratorgenerators.
Learnthatyourimplementationofaniteratorobject(aniteratorclass)can
"refresh"itselfandlearnatleastonewaytodothis.
Definitions:
IteratorAnditeratorisanobjectthatsatisfies(implements)theiteratorprotocol.
IteratorprotocolAnobjectimplementstheiteratorprotocolifitimplementsboth
anext()andan__iter__()methodwhichsatisfytheserules:(1)the
__iter__()methodmustreturntheiterator;(2)thenext()methodshould
returnthenextitemtobeiteratedoverandwhenfinished(therearenomore
items)shouldraisetheStopIterationexception.Theiteratorprotocolis
describedatIteratorTypes
http://docs.python.org/library/stdtypes.html#iteratortypes.
IteratorclassAclassthatimplements(satisfies)theiteratorprotocol.In
particular,theclassimplementsnext()and__iter__()methodsas
describedaboveandinIteratorTypes
(Iterator)generatorfunctionAfunction(ormethod)which,whencalled,returns
aniteratorobject,thatis,anobjectthatsatisfiestheiteratorprotocol.Afunction
containingayieldstatementautomaticallybecomesagenerator.
GeneratorexpressionAnexpressionwhichproducesaniteratorobject.
Generatorexpressionshaveaformsimilartoalistcomprehension,butare
enclosedinparenthesesratherthansquarebrackets.Seeexamplebelow.
Afewadditionalbasicpoints:
Afunctionthatcontainsayieldstatementisageneratorfunction.Whencalled,it
returnsaniterator,thatis,anobjectthatprovidesnext()and__iter__()
methods.
Theiteratorprotocolisdescribedhere:PythonStandardLibrary:IteratorTypes
Page97
APythonBook
Aclassthatdefinesbothanext()methodanda__iter__()methodsatisfies
theiteratorprotocol.So,instancesofsuchaclasswillbeiterators.
Pythonprovidesavarietyofwaystoproduce(implement)iterators.Thissection
describesafewofthoseways.Youshouldalsolookattheiter()builtin
function,whichisdescribedinThePythonStandardLibrary:BuiltinFunctions:
iter()http://docs.python.org/library/functions.html#iter.
Aniteratorcanbeusedinaniteratorcontext,forexampleinaforstatement,ina
listcomprehension,andinageneratorexpression.Whenaniteratorisusedinan
iteratorcontext,theiteratorproducesitsvalues.
Thissectionattemptstoprovideexamplesthatillustratethegenerator/iteratorpattern.
Whyisthisimportant?
Oncemastered,itisasimple,convenient,andpowerfulprogrammingpattern.
Ithasmanyandpervasiveuses.
Ithelpstolexicallyseparatetheproducercodefromtheconsumercode.Doingso
makesiteasiertolocateproblemsandtomodifyorfixcodeinawaythatis
localizedanddoesnothaveunwantedsideeffects.
Implementingyourowniterators(andgenerators)enablesyoutodefineyourown
abstractsequences,thatis,sequenceswhosecompositionaredefinedbyyour
computationsratherthanbytheirpresenceinacontainer.Infact,youriteratorcan
calculateorretrievevaluesaseachoneisrequested.
ExamplesTheremainderofthissectionprovidesasetofexampleswhichimplement
anduseiterators.
2.3.1ExampleAgeneratorfunction
Thisfunctioncontainsayieldstatement.Therefore,whenwecallit,itproducesan
iterator:
defgenerateItems(seq):
foriteminseq:
yield'item:%s'%item
anIter=generateItems([])
print'dir(anIter):',dir(anIter)
anIter=generateItems([111,222,333])
forxinanIter:
printx
anIter=generateItems(['aaa','bbb','ccc'])
printanIter.next()
printanIter.next()
printanIter.next()
printanIter.next()
Runningthisexampleproducesthefollowingoutput:
Page98
APythonBook
dir(anIter):['__class__','__delattr__','__doc__',
'__getattribute__',
'__hash__','__init__','__iter__','__new__','__reduce__',
'__reduce_ex__','__repr__','__setattr__','__str__','gi_frame',
'gi_running','next']
item:111
item:222
item:333
item:aaa
item:bbb
item:ccc
File"iterator_generator.py",line14,in?
printanIter.next()
StopIteration
Notesandexplanation:
Thevaluereturnedbythecalltothegenerator(function)isaniterator.Itobeys
theiteratorprotocol.Thatis,dir(anIter)showsthatithasboth
__iter__()andnext()methods.
Becausethisobjectisaniterator,wecanuseaforstatementtoiterateoverthe
valuesreturnedbythegenerator.
Wecanalsogetitsvaluesbyrepeatedlycallingthenext()method,untilit
raisestheStopIterationexception.Thisabilitytocallthenextmethodenablesus
topasstheiteratorobjectaroundandgetvaluesatdifferentlocationsinourcode.
Oncewehaveobtainedallthevaluesfromaniterator,itis,ineffect,"empty"or
"exhausted".Theiteratorprotocol,infact,specifiesthatonceaniteratorraisesthe
StopIterationexception,itshouldcontinuetodoso.Anotherwaytosaythisis
thatthereisno"rewind"operation.But,youcancallthethegeneratorfunction
againtogeta"fresh"iterator.
Analternativeandperhapssimplerwaytocreateaninteratoristouseagenerator
expression.Thiscanbeusefulwhenyoualreadyhaveacollectionoriteratortowork
with.
Thenfollowingexampleimplementsafunctionthatreturnsageneratorobject.Theeffect
istogeneratetheobjectsinacollectionwhichexcludingitemsinasepartecollection:
DATA=[
'lemon',
'lime',
'grape',
'apple',
'pear',
'watermelon',
'canteloupe',
'honeydew',
'orange',
Page99
APythonBook
'grapefruit',
]
defmake_producer(collection,excludes):
gen=(itemforitemincollectionifitemnotinexcludes)
returngen
deftest():
iter1=make_producer(DATA,('apple','orange','honeydew',))
print'%s'%iter1
forfruitiniter1:
printfruit
test()
Whenrun,thisexampleproducesthefollowing:
$pythonworkbook063.py
<generatorobject<genexpr>at0x7fb3d0f1bc80>
lemon
lime
grape
pear
watermelon
canteloupe
grapefruit
Notes:
Ageneratorexpressionlooksalmostlikealistcomprehension,butissurrounded
byparenthesesratherthansquarebrackets.Formoreonlistcomprehensionssee
sectionExampleAlistcomprehension.
Themake_producerfunctionreturnstheobjectproducedbythegenerator
expression.
2.3.2ExampleAclasscontainingageneratormethod
Eachtimethismethodiscalled,itproducesa(new)iteratorobject.Thismethodis
analogoustotheiterkeysanditervaluesmethodsinthedictionarybuiltinobject:
#
#Aclassthatprovidesaniteratorgeneratormethod.
#
classNode:
def__init__(self,name='<noname>',value='<novalue>',
children=None):
self.name=name
self.value=value
ifchildrenisNone:
self.children=[]
Page100
APythonBook
else:
defset_name(self,name):self.name=name
defget_name(self):returnself.name
defset_value(self,value):self.value=value
defget_value(self):returnself.value
defiterchildren(self):
yieldchild
#
#Printinformationonthisnodeandwalkoverallchildrenand
#grandchildren...
defwalk(self,level=0):
print'%sname:%svalue:%s'%(
get_filler(level),self.get_name(),self.get_value(),)
forchildinself.iterchildren():
child.walk(level+1)
#
#Anfunctionthatistheequivalentofthewalk()methodin
#classNode.
#
defwalk(node,level=0):
print'%sname:%svalue:%s'%(
get_filler(level),node.get_name(),node.get_value(),)
forchildinnode.iterchildren():
walk(child,level+1)
defget_filler(level):
return''*level
deftest():
a7=Node('gilbert','777')
a6=Node('fred','666')
a5=Node('ellie','555')
a4=Node('daniel','444')
a3=Node('carl','333',[a4,a5])
a2=Node('bill','222',[a6,a7])
a1=Node('alice','111',[a2,a3])
#Usethewalkmethodtowalktheentiretree.
print'Usingthemethod:'
a1.walk()
print'='*30
#Usethewalkfunctiontowalktheentiretree.
print'Usingthefunction:'
walk(a1)
test()
Usingthemethod:
name:alicevalue:111
Page101
APythonBook
name:billvalue:222
name:fredvalue:666
name:gilbertvalue:777
name:carlvalue:333
name:danielvalue:444
name:ellievalue:555
==============================
Usingthefunction:
name:alicevalue:111
name:billvalue:222
name:fredvalue:666
name:gilbertvalue:777
name:carlvalue:333
name:danielvalue:444
name:ellievalue:555
Thisclasscontainsamethoditerchildrenwhich,whencalled,returnsaniterator.
Theyieldstatementinthemethoditerchildrenmakesitintoagenerator.
Theyieldstatementreturnsoneitemeachtimeitisreached.Thenexttimethe
iteratorobjectis"called"itresumesimmediatelyaftertheyieldstatement.
Afunctionmayhaveanynumberofyieldstatements.
Aforstatementwilliterateoveralltheitemsproducedbyaniteratorobject.
Thisexampleshowstwowaystousethegenerator,specifically:(1)thewalk
methodintheclassNodeand(2)thewalkfunction.Bothcallthegenerator
iterchildrenandbothdoprettymuchthesamething.
2.3.3ExampleAniteratorclass
Thisclassimplementstheiteratorprotocol.Therefore,instancesofthisclassareiterators.
Thepresenceofthenext()and__iter__()methodsmeansthatthisclass
implementstheiteratorprotocolandmakesinstancesofthisclassiterators.
Notethatwhenaniteratoris"exhausted"it,normally,cannotbereusedtoiterateoverthe
sequence.However,inthisexample,weprovidearefreshmethodwhichenablesusto
"rewind"andreusetheiteratorinstance:
#
#Aniteratorclassthatdoes*not*use``yield``.
#Thisiteratorproduceseveryotheriteminasequence.
#
classIteratorExample:
def__init__(self,seq):
self.seq=seq
self.idx=0
defnext(self):
self.idx+=1
ifself.idx>=len(self.seq):
Page102
APythonBook
raiseStopIteration
value=self.seq[self.idx]
self.idx+=1
returnvalue
def__iter__(self):
returnself
defrefresh(self):
self.idx=0
deftest_iteratorexample():
a=IteratorExample('edcba')
forxina:
printx
print''
a.refresh()
forxina:
printx
print'='*30
a=IteratorExample('abcde')
try:
printa.next()
printa.next()
printa.next()
printa.next()
printa.next()
printa.next()
exceptStopIteration,e:
print'stopping',e
test_iteratorexample()
d
b
d
b
==============================
b
d
stopping
Thenextmethodmustkeeptrackofwhereitisandwhatitemitshouldproduce
next.
Alert:TheiteratorprotocolhaschangedslightlyinPython3.0.Inparticular,the
next()methodhasbeenrenamedto__next__().See:PythonStandard
Library:IteratorTypes
http://docs.python.org/3.0/library/stdtypes.html#iteratortypes.
Page103
APythonBook
2.3.4ExampleAniteratorclassthatusesyield
Theremaybetimeswhenthenextmethodiseasierandmorestraightforwardto
implementusingyield.Ifso,thenthisclassmightserveasanmodel.Ifyoudonotfeel
theneedtodothis,thenyoushouldignorethisexample:
#
#Aniteratorclassthatuses``yield``.
#Thisiteratorproduceseveryotheriteminasequence.
#
classYieldIteratorExample:
def__init__(self,seq):
self.seq=seq
self.iterator=self._next()
self.next=self.iterator.next
def_next(self):
flag=0
forxinself.seq:
ifflag:
flag=0
yieldx
else:
flag=1
def__iter__(self):
returnself.iterator
defrefresh(self):
deftest_yielditeratorexample():
a=YieldIteratorExample('edcba')
forxina:
printx
print''
a.refresh()
forxina:
printx
print'='*30
a=YieldIteratorExample('abcde')
try:
printa.next()
printa.next()
printa.next()
printa.next()
printa.next()
printa.next()
exceptStopIteration,e:
print'stopping',e
test_yielditeratorexample()
Page104
APythonBook
d
b
d
b
==============================
b
d
stopping
Becausethe_nextmethodusesyield,callingit(actually,callingtheiterator
objectitproduces)inaniteratorcontextcausesittobe"resumed"immediately
aftertheyieldstatement.Thisreducesbookkeepingabit.
However,withthisstyle,wemustexplicitlyproduceaniterator.Wedothisby
callingthe_nextmethod,whichcontainsayieldstatement,andisthereforea
generator.Thefollowingcodeinourconstructor(__init__)completesthe
setupofourclassasaniteratorclass:
Rememberthatweneedboth__iter__()andnext()methodsin
YieldIteratorExampletosatisfytheiteratorprotocol.The__iter__()
methodisalreadythereandtheabovecodeintheconstructorcreatesthenext()
method.
2.3.5ExampleAlistcomprehension
Alistcomprehensionlooksabitlikeaniterator,butitproducesalist.See:ThePython
LanguageReference:Listdisplays
http://docs.python.org/reference/expressions.html#listdisplaysformoreonlist
comprehensions.
Hereisanexample:
In[4]:deff(x):
...:returnx*3
...:
In[5]:list1=[11,22,33]
In[6]:list2=[f(x)forxinlist1]
In[7]:printlist2
[33,66,99]
2.3.6ExampleAgeneratorexpression
Ageneratorexpressionlooksquitesimilartoalistcomprehension,butisenclosedin
Page105
APythonBook
parenthesesratherthansquarebrackets.Unlikealistcomprehension,agenerator
expressiondoesnotproducealist;itproducesangeneratorobject.Ageneratorobjectis
aniterator.
Formoreongeneratorexpressions,seeThePythonLanguageReference:Generator
expressionshttp://docs.python.org/reference/expressions.html#generatorexpressions.
Thefollowingexampleusesageneratorexpressiontoproduceaniterator:
mylist=range(10)
deff(x):
returnx*3
genexpr=(f(x)forxinmylist)
forxingenexpr:
printx
Thegeneratorexpression(f(x)forxinmylist)producesaniteratorobject.
Noticethatwecanusetheiteratorobjectlaterinourcode,cansaveitinadata
structure,andcanpassittoafunction.
2.4UnitTests
UnittestandthePythonunittestframeworkprovideaconvenientwaytodefineandrun
teststhatensurethataPythonapplicationproducesspecifiedresults.
Thissection,whileitwillnotattempttoexplaineverythingabouttheunittestframework,
willprovideexamplesofseveralstraightforwardwaystoconstructandruntests.
Someassumptions:
Wearegoingtodevelopasoftwareprojectincrementally.Wewillnotimplement
andreleaseallatonce.Therefore,eachtimeweaddtoourexistingcodebase,we
needawaytoverifythatouradditions(andfixes)havenotcausednewproblems
inoldcode.
Addingnewcodetoexistingcodewillcauseproblems.Weneedtobeableto
check/testforthoseproblemsateachstep.
Asweaddcode,weneedtobeabletoaddtestsforthatnewcode,too.
2.4.1Definingunittests
2.4.1.1Createatestclass.
Inthetestclass,implementanumberofmethodstoperformyourtests.Nameyourtest
Page106
APythonBook
methodswiththeprefix"test".Hereisanexample:
importunittest
classMyTest(unittest.TestCase):
deftest_one(self):
#sometestcode
pass
deftest_two(self):
#sometestcode
pass
Createatestharness.Hereisanexample:
importunittest
#makethetestsuite.
defsuite():
testsuite=loader.loadTestsFromTestCase(MyTest)
returntestsuite
#Makethetestsuite;runthetests.
deftest():
testsuite=suite()
result=runner.run(testsuite)
Hereisamorecompleteexample:
importsys,StringIO,string
importunittest
importwebserv_example_heavy_sub
#Acomparisonfunctionforcaseinsenstivesorting.
defmycmpfunc(arg1,arg2):
returncmp(string.lower(arg1),string.lower(arg2))
classXmlTest(unittest.TestCase):
deftest_import_export1(self):
inFile=file('test1_in.xml','r')
inContent=inFile.read()
inFile.close()
doc=webserv_example_heavy_sub.parseString(inContent)
outFile=StringIO.StringIO()
outFile.write('<?xmlversion="1.0"?>\n')
doc.export(outFile,0)
outContent=outFile.getvalue()
outFile.close()
self.failUnless(inContent==outContent)
#makethetestsuite.
defsuite():
Page107
APythonBook
#Changethetestmethodprefix:test>trial.
#loader.testMethodPrefix='trial'
#Changethecomparisonfunctionthatdeterminestheorderof
tests.
#loader.sortTestMethodsUsing=mycmpfunc
testsuite=loader.loadTestsFromTestCase(XmlTest)
returntestsuite
#Makethetestsuite;runthetests.
deftest_main():
testsuite=suite()
result=runner.run(testsuite)
if__name__=="__main__":
test_main()
Runningtheabovescriptproducesthefollowingoutput:
test_import_export(__main__.XmlTest)...ok
Ran1testin0.035s
OK
Afewnotesonthisexample:
Thisexampleteststheabilitytoparseanxmldocumenttest1_in.xmlandexport
thatdocumentbacktoXML.ThetestsucceedsiftheinputXMLdocumentand
theexportedXMLdocumentarethesame.
ThecodewhichisbeingtestedparsesanXMLdocumentreturnedbyarequestto
AmazonWebservices.YoucanlearnmoreaboutAmazonWebservicesat:
http://www.amazon.com/webservices.ThiscodewasgeneratedfromanXML
SchemadocumentbygenerateDS.py.Soweareineffect,testinggenerateDS.py.
YoucanfindgenerateDS.pyat:
http://http://www.davekuhlman.org/#generatedspy.
Testingforsuccess/failureandreportingfailuresUsethemethodslistedat
http://www.python.org/doc/current/lib/testcaseobjects.htmltotestforandreport
successandfailure.Inourexample,weused"self.failUnless(inContent==
outContent)"toensurethatthecontentweparsedandthecontentthatwe
exportedwerethesame.
Addadditionaltestsbyaddingmethodswhosenameshavetheprefix"test".If
youpreferadifferentprefixfortestsnames,addsomethinglikethefollowingto
theabovescript:
loader.testMethodPrefix='trial'
Page108
APythonBook
Bydefault,thetestsarerunintheorderoftheirnamessortedbythecmp
function.So,ifneeded,youcancontroltheorderofexecutionoftestsby
selectingtheirnames,forexample,usingnamesliketest_1_checkderef,
test_2_checkcalc,etc.Or,youcanchangethecomparisonfunctionbyadding
somethinglikethefollowingtotheabovescript:
loader.sortTestMethodsUsing=mycmpfunc
Asabitofmotivationforcreatingandusingunittests,whiledevelopingthisexample,I
discoveredseveralerrors(ormaybe"specialfeatures")ingenerateDS.py.
2.5ExtendingandembeddingPython
2.5.1Introductionandconcepts
Extendingvs.embeddingTheyaredifferentbutrelated:
ExtendingPythonmeanstoimplementanextensionmoduleoranextensiontype.
AnextensionmodulecreatesanewPythonmodulewhichisimplementedin
C/C++.FromPythoncode,anextensionmoduleappearstobejustlikeamodule
implementedinPythoncode.AnextensiontypecreatesanewPython(builtin)
typewhichisimplementedinC/C++.FromPythoncode,anextensiontype
appearstobejustlikeabuiltintype.
EmbeddingPython,bycontrast,istoputthePythoninterpreterwithinan
application(i.e.linkitin)sothattheapplicationcanrunPythonscripts.The
scriptscanbeexecutedortriggeredinavarietyofways,e.g.theycanbeboundto
keysonthekeyboardortomenuitems,theycanbetriggeredbyexternalevents,
etc.Usually,inordertomaketheembeddedPythoninterpreteruseful,Pythonis
alsoextendedwithfunctionsfromtheembeddingapplication,sothatthescripts
cancallfunctionsthatareimplementedbytheembeddingC/C++application.
DocumentationThetwoimportantsourcesforinformationaboutextendingand
embeddingarethefollowing:
ExtendingandEmbeddingthePythonInterpreter
http://www.python.org/doc/current/ext/ext.html
Python/CAPIReferenceManual
http://www.python.org/doc/current/api/api.html
Typesofextensions:
ExtensionmodulesFromthePythonside,itappearstobeaPythonmodule.
Usuallyitexportsfunctions.
ExtensiontypesUsedtoimplementanewPythondatatype.
ExtensionclassesFromthePythonside,itappearstobeaclass.
Page109
APythonBook
ToolsThereareseveraltoolsthatsupportthedevelopmentofPythonextensions:
SWIGLearnaboutSWIGat:http://www.swig.org
PyrexLearnaboutPyrexat:
http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/
ThereisalsoCython,whichseemstobeanadvancedversionof,oratleastan
alternativetoPyrex.See:CythonCExtensionsforPython
http://www.cython.org/
2.5.2Extensionmodules
WritinganextensionmodulebyhandWhattodo:
Createthe"init"functionThenameofthisfunctionmustbe"init"followedby
thenameofthemodule.Everyextensionmodulemusthavesuchafunction.
CreatethefunctiontableThistablemapsfunctionnames(referencedfrom
Pythoncode)tofunctionpointers(implementedinC/C++).
Implementeachwrapperfunction.
ImplementingawrapperfunctionWhattodo:
1. CapturetheargumentswithPyArg_ParseTuple.Theformatstringspecifieshow
argumentsaretobeconvertedandcaptured.See1.7ExtractingParametersin
ExtensionFunctions.Herearesomeofthemostcommonlyusedtypes:
Use"i","s","f",etctoconvertandcapturesimpletypessuchasintegers,
strings,floats,etc.
Use"O"togetapointertoPython"complex"typessuchaslists,tuples,
dictionaries,etc.
Useitemsinparenthesestocaptureandunpacksequences(e.g.listsand
tuples)offixedlength.Example:
if(!PyArg_ParseTuple(args,"(ii)(ii)",&x,&y,
&width,&height))
{
returnNULL;
}/*if*/
Asamplecallmightbe:
lowerLeft=(x1,y1)
extent=(width1,height1)
scan(lowerLeft,extent)
Use":aName"(colon)attheendoftheformatstringtoprovideafunction
nameforerrormessages.Example:
if(!PyArg_ParseTuple(args,"O:setContentHandler",
&pythonInstance))
{
Page110
APythonBook
returnNULL;
}/*if*/
Use";anerrormessage"(semicolon)attheendoftheformatstringtoprovide
astringthatreplacesthedefaulterrormessage.
Docsareavailableat:http://www.python.org/doc/current/ext/parseTuple.html.
2. Writethelogic.
3. HandleerrorsandexceptionsYouwillneedtounderstandhowto(1)clearing
errorsandexceptionsand(2)Raiseerrors(exceptions).
ManyfunctionsinthePythonCAPIraiseexceptions.Youwillneedtocheck
forandcleartheseexceptions.Hereisanexample:
char*message;
intmessageNo;
message=NULL;
messageNo=1;
/*Istheargumentastring?
*/
if(!PyArg_ParseTuple(args,"s",&message))
{
/*It'snotastring.Cleartheerror.
*Thentrytogetamessagenumber(an
integer).
*/
PyErr_Clear();
if(!PyArg_ParseTuple(args,"i",&messageNo))
{
o
o
o
YoucanalsoraiseexceptionsinyourCcodethatcanbecaught(ina
"try:except:"block)backinthecallingPythoncode.Hereisanexample:
if(n==0)
{
PyErr_SetString(PyExc_ValueError,"Valuemust
notbezero");
returnNULL;
}
SeeInclude/pyerrors.hinthePythonsourcedistributionformore
exception/errortypes.
And,youcantestwhetherafunctioninthePythonCAPIthatyouhavecalled
hasraisedanexception.Forexample:
if(PyErr_Occurred())
{
/*Anexceptionwasraised.
*Dosomethingaboutit.
Page111
APythonBook
*/
o
o
o
Formoredocumentationonerrorsandexceptions,see:
http://www.python.org/doc/current/api/exceptionHandling.html.
4. Createandreturnavalue:
ForeachbuiltinPythontypethereisasetofAPIfunctionstocreateand
manipulateit.Seethe"Python/CAPIReferenceManual"foradescriptionof
thesefunctions.Forexample,see:
http://www.python.org/doc/current/api/intObjects.html
http://www.python.org/doc/current/api/stringObjects.html
http://www.python.org/doc/current/api/tupleObjects.html
http://www.python.org/doc/current/api/listObjects.html
http://www.python.org/doc/current/api/dictObjects.html
Etc.
ThereferencecountYouwillneedtofollowPython'srulesforreference
countingthatPythonusestogarbagecollectobjects.Youcanlearnabout
theserulesathttp://www.python.org/doc/current/ext/refcounts.html.Youwill
notwantPythontogarbagecollectobjectsthatyoucreatetooearlyortoolate.
WithrespecttoPythonobjectscreatedwiththeabovefunctions,thesenew
objectsareownedandmaybepassedbacktoPythoncode.However,there
aresituationswhereyourC/C++codewillnotautomaticallyownareference,
forexamplewhenyouextractanobjectfromacontainer(alist,tuple,
dictionary,etc).Inthesecasesyoushouldincrementthereferencecountwith
Py_INCREF.
2.5.3SWIG
Note:OurdiscussionandexamplesareforSWIGversion1.3
SWIGwilloftenenableyoutogeneratewrappersforfunctionsinanexistingCfunction
library.SWIGdoesnotunderstandeverythinginCheaderfiles.Butitdoesafairly
impressivejob.Youshouldtryitfirstbeforeresortingtothehardworkofwriting
wrappersbyhand.
MoreinformationonSWIGisathttp://www.swig.org.
Herearesomestepsthatyoucanfollow:
1. CreateaninterfacefileEvenwhenyouarewrappingfunctionsdefinedinan
existingheaderfile,creatinganinterfacefileisagoodidea.Includeyourexisting
headerfileintoit,thenaddwhateverelseyouneed.Hereisanextremelysimple
exampleofaSWIGinterfacefile:
Page112
APythonBook
%moduleMyLibrary
%{
#include"MyLibrary.h"
%}
%include"MyLibrary.h"
Comments:
The"%{"and"%}"bracketsaredirectivestoSWIG.Theysay:"Addthecode
betweenthesebracketstothegeneratedwrapperfilewithoutprocessingit.
The"%include"statementsays:"Copythefileintotheinterfacefilehere.In
effect,youareaskingSWIGtogeneratewrappersforallthefunctionsinthis
headerfile.Ifyouwantwrappersforonlysomeofthefunctionsinaheader
file,thencopyorreproducefunctiondeclarationsforthedesiredfunctions
here.Anexample:
%moduleMyLibrary
%{
#include"MyLibrary.h"
%}
intcalcArea(intwidth,intheight);
intcalcVolume(intradius);
Thisexamplewillgeneratewrappersforonlytwofunctions.
YoucanfindmoreinformationaboutthedirectivesthatareusedinSWIG
interfacefilesintheSWIGUserManual,inparticularat:
http://www.swig.org/Doc1.3/Preprocessor.html
http://www.swig.org/Doc1.3/Python.html
2. Generatethewrappers:
swigpythonMyLibrary.i
3. Compileandlinkthelibrary.OnLinux,youcanusesomethinglikethefollowing:
gcccMyLibrary.c
gcccI/usr/local/include/python2.3MyLibrary_wrap.c
gccsharedMyLibrary.oMyLibrary_wrap.oo
_MyLibrary.so
Notethatweproduceasharedlibrarywhosenameisthemodulenameprefixed
withanunderscore.SWIGalsogeneratesa.pyfile,withouttheleading
underscore,whichwewillimportfromourPythoncodeandwhich,inturn,
importsthesharedlibrary.
4. Usetheextensionmoduleinyourpythoncode:
Python2.3b1(#1,Apr252003,20:36:09)
[GCC2.95.420011002(Debianprerelease)]onlinux2
Page113
APythonBook
Type"help","copyright","credits"or"license"for
moreinformation.
>>>importMyLibrary
>>>MyLibrary.calcArea(4.0,5.0)
20.0
Hereisamakefilethatwillexecuteswigtogeneratewrappers,thencompileandlinkthe
extension.
CFLAGS=I/usr/local/include/python2.3
all:_MyLibrary.so
_MyLibrary.so:MyLibrary.oMyLibrary_wrap.o
gccsharedMyLibrary.oMyLibrary_wrap.oo_MyLibrary.so
MyLibrary.o:MyLibrary.c
gcccMyLibrary.coMyLibrary.o
MyLibrary_wrap.o:MyLibrary_wrap.c
gccc${CFLAGS}MyLibrary_wrap.coMyLibrary_wrap.o
MyLibrary_wrap.c:MyLibrary.i
clean:
rmfMyLibrary.pyMyLibrary.oMyLibrary_wrap.c
MyLibrary_wrap.o_MyLibrary.so
Hereisanexampleofrunningthismakefile:
$makefMyLibrary_makefileclean
rmfMyLibrary.pyMyLibrary.oMyLibrary_wrap.c\
MyLibrary_wrap.o_MyLibrary.so
$makefMyLibrary_makefile
gcccMyLibrary.coMyLibrary.o
gcccI/usr/local/include/python2.3MyLibrary_wrap.co
MyLibrary_wrap.o
gccsharedMyLibrary.oMyLibrary_wrap.oo_MyLibrary.so
And,hereareCsourcefilesthatcanbeusedinourexample.
MyLibrary.h:
/*MyLibrary.h
*/
Page114
APythonBook
floatcalcArea(floatwidth,floatheight);
floatcalcVolume(floatradius);
intgetVersion();
intgetMode();
MyLibrary.c:
/*MyLibrary.c
*/
floatcalcArea(floatwidth,floatheight)
{
return(width*height);
}
floatcalcVolume(floatradius)
{
return(3.14*radius*radius);
}
intgetVersion()
{
return123;
}
intgetMode()
{
return1;
}
2.5.4Pyrex
PyrexisausefultoolforwritingPythonextensions.BecausethePyrexlanguageis
similartoPython,writingextensionsinPyrexiseasierthandoingsoinC.Cython
appearstobetheanewerversionofPyrex.
MoreinformationisonPyrexandCythonisat:
Pyrexhttp://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/
CythonCExtensionsforPythonhttp://www.cython.org/
HereisasimplefunctiondefinitioninPyrex:
#python_201_pyrex_string.pyx
importstring
defformatString(objects1,objects2):
s1=string.strip(s1)
s2=string.strip(s2)
Page115
APythonBook
s3='<<%s||%s>>'%(s1,s2)
s4=s3*4
returns4
And,hereisamakefile:
CFLAGS=DNDEBUGO3WallWstrictprototypesfPIC\
I/usr/local/include/python2.3
all:python_201_pyrex_string.so
python_201_pyrex_string.so:python_201_pyrex_string.o
gccsharedpython_201_pyrex_string.oo
python_201_pyrex_string.so
python_201_pyrex_string.o:python_201_pyrex_string.c
gccc${CFLAGS}python_201_pyrex_string.co
python_201_pyrex_string.o
python_201_pyrex_string.c:python_201_pyrex_string.pyx
pyrexcpython_201_pyrex_string.pyx
clean:
rmfpython_201_pyrex_string.sopython_201_pyrex_string.o\
python_201_pyrex_string.c
Hereisanotherexample.Inthisone,onefunctioninthe.pyxfilecallsanother.Hereis
theimplementationfile:
#python_201_pyrex_primes.pyx
defshowPrimes(intkmax):
plist=primes(kmax)
forpinplist:
print'prime:%d'%p
cdefprimes(intkmax):
cdefintn,k,i
cdefintp[1000]
result=[]
ifkmax>1000:
kmax=1000
k=0
n=2
whilek<kmax:
i=0
whilei<kandn%p[i]<>0:
i=i+1
ifi==k:
p[k]=n
k=k+1
result.append(n)
n=n+1
Page116
APythonBook
returnresult
#CFLAGS=DNDEBUGgO3WallWstrictprototypesfPIC#
I/usr/local/include/python2.3CFLAGS=DNDEBUG
I/usr/local/include/python2.3
all:python_201_pyrex_primes.so
python_201_pyrex_primes.so:python_201_pyrex_primes.o
gccsharedpython_201_pyrex_primes.oopython_201_pyrex_primes.so
python_201_pyrex_primes.o:python_201_pyrex_primes.c
gccc${CFLAGS}python_201_pyrex_primes.copython_201_pyrex_primes.o
python_201_pyrex_primes.c:python_201_pyrex_primes.pyx
pyrexcpython_201_pyrex_primes.pyx
clean:
rmfpython_201_pyrex_primes.sopython_201_pyrex_primes.o
python_201_pyrex_primes.c
Hereistheoutputfromrunningthemakefile:
$makefpython_201_pyrex_makeprimesclean
rmfpython_201_pyrex_primes.sopython_201_pyrex_primes.o\
python_201_pyrex_primes.c
$makefpython_201_pyrex_makeprimes
pyrexcpython_201_pyrex_primes.pyx
gcccDNDEBUGI/usr/local/include/python2.3
python_201_pyrex_primes.copython_201_pyrex_primes.o
gccsharedpython_201_pyrex_primes.oopython_201_pyrex_primes.so
Hereisaninteractiveexampleofitsuse:
$python
Python2.3b1(#1,Apr252003,20:36:09)
information.
>>>importpython_201_pyrex_primes
>>>dir(python_201_pyrex_primes)
['__builtins__','__doc__','__file__','__name__','showPrimes']
>>>python_201_pyrex_primes.showPrimes(5)
prime:2
prime:3
prime:5
prime:7
Page117
APythonBook
prime:11
ThisnextexampleshowshowtousePyrextoimplementanewextensiontype,thatisa
newPythonbuiltintype.Noticethattheclassisdeclaredwiththecdefkeyword,which
tellsPyrextogeneratetheCimplementationofatypeinsteadofaclass.
Hereistheimplementationfile:
#python_201_pyrex_clsprimes.pyx
"""Animplementationofprimeshandlingclass
forademonstrationofPyrex.
"""
cdefclassPrimes:
"""Aclasscontainingfunctionsfor
handlingprimes.
"""
defshowPrimes(self,intkmax):
"""Showarangeofprimes.
Usethemethodprimes()togeneratetheprimes.
"""
plist=self.primes(kmax)
forpinplist:
print'prime:%d'%p
defprimes(self,intkmax):
"""Generatetheprimesintherange0kmax.
"""
cdefintn,k,i
cdefintp[1000]
result=[]
ifkmax>1000:
kmax=1000
k=0
n=2
whilek<kmax:
i=0
whilei<kandn%p[i]<>0:
i=i+1
ifi==k:
p[k]=n
k=k+1
result.append(n)
n=n+1
returnresult
CFLAGS=DNDEBUGI/usr/local/include/python2.3
all:python_201_pyrex_clsprimes.so
Page118
APythonBook
python_201_pyrex_clsprimes.so:python_201_pyrex_clsprimes.o
gccsharedpython_201_pyrex_clsprimes.oo
python_201_pyrex_clsprimes.so
python_201_pyrex_clsprimes.o:python_201_pyrex_clsprimes.c
gccc${CFLAGS}python_201_pyrex_clsprimes.co
python_201_pyrex_clsprimes.o
python_201_pyrex_clsprimes.c:python_201_pyrex_clsprimes.pyx
pyrexcpython_201_pyrex_clsprimes.pyx
clean:
rmfpython_201_pyrex_clsprimes.so
python_201_pyrex_clsprimes.o\
python_201_pyrex_clsprimes.c
Hereisoutputfromrunningthemakefile:
$makefpython_201_pyrex_makeclsprimesclean
rmfpython_201_pyrex_clsprimes.sopython_201_pyrex_clsprimes.o\
python_201_pyrex_clsprimes.c
$makefpython_201_pyrex_makeclsprimes
pyrexcpython_201_pyrex_clsprimes.pyx
gcccDNDEBUGI/usr/local/include/python2.3
python_201_pyrex_clsprimes.copython_201_pyrex_clsprimes.o
gccsharedpython_201_pyrex_clsprimes.oo
python_201_pyrex_clsprimes.so
Andhereisaninteractiveexampleofitsuse:
$python
Python2.3b1(#1,Apr252003,20:36:09)
information.
>>>importpython_201_pyrex_clsprimes
>>>dir(python_201_pyrex_clsprimes)
['Primes','__builtins__','__doc__','__file__','__name__']
>>>primes=python_201_pyrex_clsprimes.Primes()
>>>dir(primes)
['__class__','__delattr__','__doc__','__getattribute__',
'__hash__',
'__init__','__new__','__reduce__','__reduce_ex__','__repr__',
'__setattr__','__str__','primes','showPrimes']
>>>primes.showPrimes(4)
prime:2
prime:3
prime:5
prime:7
DocumentationAlsonoticethatPyrexpreservesthedocumentationforthemodule,the
class,andthemethodsintheclass.Youcanshowthisdocumentationwithpydoc,as
Page119
APythonBook
follows:
$pydocpython_201_pyrex_clsprimes
Or,inPythoninteractivemode,use:
$python
Python2.3b1(#1,Apr252003,20:36:09)
information.
>>>importpython_201_pyrex_clsprimes
>>>help(python_201_pyrex_clsprimes)
2.5.5SWIGvs.Pyrex
ChooseSWIGwhen:
YoualreadyhaveanexistingCorC++implementationofthecodeyouwantto
callfromPython.InthiscaseyouwantSWIGtogeneratethewrappers.Butnote
thatCythonpromisestoenableyoutoquicklywrapandcallfunctions
implementedinC.
YouwanttowritetheimplementationinCorC++byhand.Perhaps,becauseyou
thinkyoucandosoquickly,forexample,orbecauseyoubelievethatyoucan
makeithighlyoptimized.Then,youwanttobeabletogeneratethePython
(extension)wrappersforitquickly.
ChoosePyrexwhen:
YoudonothaveaC/C++implementationandyouwantaneasierwaytowrite
thatCimplementation.WritingPyrexcode,whichisalotlikePython,iseasier
thanwritingCorC++codebyhand).
YoustarttowritetheimplementationinC,thenfindthatitrequireslotsofcallsto
thePythonCAPI,andyouwanttoavoidhavingtolearnhowtodothat.
2.5.6Cython
HereisasimpleexamplethatusesCythontowrapafunctionimplementedinC.
FirsttheCheaderfile:
/*test_c_lib.h*/
intcalculate(intwidth,intheight);
And,theCimplementationfile:
/*test_c_lib.c*/
Page120
APythonBook
#include"test_c_lib.h"
intcalculate(intwidth,intheight)
{
intresult;
result=width*height*3;
returnresult;
}
HereisaCythonfilethatcallsourCfunction:
#test_c.pyx
#DeclaretheexternalCfunction.
cdefexternfrom"test_c_lib.h":
intcalculate(intwidth,intheight)
deftest(w,h):
#CalltheexternalCfunction.
result=calculate(w,h)
print'resultfromcalculate:%d'%result
Wecancompileourcodeusingthisscript(onLinux):
#!/bin/bashx
cythontest_c.pyx
gcccfPICI/usr/local/include/python2.6otest_c.otest_c.c
gcccfPICI/usr/local/include/python2.6otest_c_lib.o
test_c_lib.c
gccsharedfPICI/usr/local/include/python2.6otest_c.so
test_c.otest_c_lib.o
HereisasmallPythonfilethatusesthewrapperthatwewroteinCython:
#run_test_c.py
importtest_c
deftest():
test_c.test(4,5)
test_c.test(12,15)
test()
And,whenwerunit,weseethefollowing:
$pythonrun_test_c.py
resultfromcalculate:60
resultfromcalculate:540
Page121
APythonBook
2.5.7Extensiontypes
ThegoalAnewbuiltindatatypeforPython.
ExistingexamplesObjects/listobject.c,Objects/stringobject.c,Objects/dictobject.c,etc
inthePythonsourcecodedistribution.
InolderversionsofthePythonsourcecodedistribution,atemplatefortheCcodewas
providedinObjects/xxobject.c.Objects/xxobject.cisnolongerincludedinthePython
sourcecodedistribution.However:
Thediscussionandexamplesforcreatingextensiontypeshavebeenexpanded.
See:ExtendingandEmbeddingthePythonInterpreter,2.DefiningNewTypes
http://docs.python.org/extending/newtypes.html.
IntheTools/framerdirectoryofthePythonsourcecodedistributionthereisan
applicationthatwillgenerateaskeletonforanextensiontypefromaspecification
objectwritteninPython.RunTools/framer/example.pytoseeitinaction.
And,youcanusePyrextogenerateanewbuiltintype.Todoso,implementa
Python/PyrexclassanddeclaretheclasswiththePyrexkeywordcdef.Infact,youmay
wanttousePyrextogenerateaminimalextensiontype,andtheneditthatgeneratedcode
toinsertandaddfunctionalitybyhand.SeethePyrexsectionforanexample.
Pyrexalsogoessomewaytowardgivingyouaccessto(existing)Cstructsandfunctions
fromPython.
2.5.8Extensionclasses
ExtensionclassestheeasywaySWIGshadowclasses.
StartwithanimplementationofaC++classanditsheaderfile.
UsethefollowingSWIGflags:
swigc++pythonmymodule.i
MoreinformationisavailablewiththeSWIGdocumentationat:
http://www.swig.org/Doc1.3/Python.html.
ExtensionclassesthePyrexwayAnalternatieistousePyrextocompileaclass
definitionthatdoesnothavethecdefkeyword.UsingcdefontheclasstellsPyrexto
generateanextensiontypeinsteadofaclass.Youwillhavetodeterminewhetheryou
wantanextensionclassoranextensiontype.
2.6Parsing
Pythonisanexcellentlanguagefortextanalysis.
Page122
APythonBook
Insomecases,simplysplittinglinesoftextintowordswillbeenough.Inthesecasesuse
string.split().
Inothercases,regularexpressionsmaybeabletodotheparsingyouneed.Ifso,seethe
sectiononregularexpressionsinthisdocument.
However,insomecases,morecomplexanalysisofinputtextisrequired.Thissection
describessomeofthewaysthatPythoncanhelpyouwiththiscomplexparsingand
analysis.
2.6.1Specialpurposeparsers
ThereareanumberofspecialpurposeparserswhichyouwillfindinthePythonstandard
library:
ConfigParserparserConfigurationfileparser
http://docs.python.org/library/configparser.html
getoptParserforcommandlineoptions
http://docs.python.org/library/getopt.html
optparseMorepowerfulcommandlineoptionparser
http://docs.python.org/library/optparse.html
urlparseParseURLsintocomponents
http://docs.python.org/library/urlparse.html
csvCSV(commaseparatedvalues)FileReadingandWriting
http://docs.python.org/library/csv.html#modulecsv
os.pathCommonpathnamemanipulations
http://docs.python.org/library/os.path.html
XMLparsersandXMLtoolsThereislotsofsupportforparsingandprocessingXML
inPython.Hereareafewplacestolookforsupport:
ThePythonstandardlibraryStructuredMarkupProcessingTools
http://docs.python.org/library/markup.html.
Inparticular,youmaybeinterestedinxml.dom.minidomLightweightDOM
implementationhttp://docs.python.org/library/xml.dom.minidom.html.
ElementTreeYoucanthinkofElementTreeasanenhancedDOM(document
objectmodel).Manyfinditeasiertousethanminidom.ElementTreeisinthe
Pythonstandardlibrary,anddocumentationishere:ElementTreeOverview
http://effbot.org/zone/elementindex.htm.
LxmlmimicstheElementTreeAPI,buthasadditionalcapabilities.Findoutabout
Lxmlatlxmlhttp://codespeak.net/lxml/index.htmlNotethatlxmlalsohas
supportforXPathandXSLT.
Dave'ssupportforPythonandXMLhttp://www.rexx.com/~dkuhlman.
Page123
APythonBook
2.6.2Writingarecursivedescentparserbyhand
Forsimplegrammars,thisisnotsohard.
Youwillneedtoimplement:
Arecognizermethodorfunctionforeachproductionruleinyourgrammar.Each
recognizermethodbeginslookingatthecurrenttoken,thenconsumesasmany
tokensasneededtorecognizeit'sownproductionrule.Itcallstherecognizer
functionsforanynonterminalsonitsrighthandside.
AtokenizerSomethingthatwillenableeachrecognizerfunctiontogettokens,
onebyone.Thereareavarietyofwaystodothis,e.g.(1)afunctionthat
producesalistoftokensfromwhichrecognizerscanpoptokens;(2)agenerator
whosenextmethodreturnsthenexttoken;etc.
Asanexample,we'llimplementarecursivedescentparserwritteninPythonforthe
followinggrammer:
Prog::=Command|CommandProg
Command::=Func_call
Func_call::=Term'('Func_call_list')'
Func_call_list::=Func_call|Func_call','Func_call_list
Term=<word>
Hereisanimplementationofarecursivedescentparserfortheabovegrammar:
"""
Arecursivedescentparserexample.
Usage:
pythonrparser.py[options]<inputfile>
Options:
Example:
pythonrparser.pymyfile.txt
Thegrammar:
Command::=Func_call
Term=<word>
"""
importsys
importstring
importtypes
importgetopt
Page124
APythonBook
#
#TousetheIPythoninteractiveshelltoinspectyourrunning
#application,uncommentthefollowinglines:
#
##fromIPython.ShellimportIPShellEmbed
##ipshell=IPShellEmbed((),
##banner='>>>>>>>>IntoIPython>>>>>>>>',
##exit_msg='<<<<<<<<OutofIPython<<<<<<<<')
#
#Thenaddthefollowinglineatthepointinyourcodewhere
#youwanttoinspectruntimevalues:
#
#ipshell('somemessagetoidentifywhereweare')
#
#Formoreinformationsee:http://ipython.scipy.org/moin/
#
#
#Constants
#
#ASTnodetypes
NoneNodeType=0
ProgNodeType=1
CommandNodeType=2
FuncCallNodeType=3
FuncCallListNodeType=4
TermNodeType=5
#Tokentypes
NoneTokType=0
LParTokType=1
RParTokType=2
WordTokType=3
CommaTokType=4
EOFTokType=5
#Dictionarytomapnodetypevaluestonodetypenames
NodeTypeDict={
NoneNodeType:'NoneNodeType',
ProgNodeType:'ProgNodeType',
CommandNodeType:'CommandNodeType',
FuncCallNodeType:'FuncCallNodeType',
FuncCallListNodeType:'FuncCallListNodeType',
TermNodeType:'TermNodeType',
}
#
#RepresentationofanodeintheAST(abstractsyntaxtree).
#
classASTNode:
def__init__(self,nodeType,*args):
self.nodeType=nodeType
Page125
APythonBook
self.children=[]
foriteminargs:
self.children.append(item)
defshow(self,level):
self.showLevel(level)
print'NodeType%s'%NodeTypeDict[self.nodeType]
level+=1
ifisinstance(child,ASTNode):
child.show(level)
eliftype(child)==types.ListType:
foriteminchild:
item.show(level)
else:
print'Child:',child
defshowLevel(self,level):
foridxinrange(level):
print'',
#
#Therecursivedescentparserclass.
#Containsthe"recognizer"methods,whichimplementthegrammar
#rules(above),onerecognizermethodforeachproductionrule.
#
classProgParser:
def__init__(self):
pass
defparseFile(self,infileName):
self.infileName=infileName
self.tokens=None
self.tokenType=NoneTokType
self.token=''
self.lineNo=1
self.infile=file(self.infileName,'r')
self.tokens=genTokens(self.infile)
try:
self.tokenType,self.token,self.lineNo=
self.tokens.next()
exceptStopIteration:
raiseRuntimeError,'Emptyfile'
result=self.prog_reco()
self.infile.close()
self.infile=None
returnresult
defparseStream(self,instream):
self.tokens=genTokens(instream,'<instream>')
try:
self.tokens.next()
Page126
APythonBook
returnresult
defprog_reco(self):
commandList=[]
while1:
result=self.command_reco()
ifnotresult:
break
commandList.append(result)
returnASTNode(ProgNodeType,commandList)
defcommand_reco(self):
ifself.tokenType==EOFTokType:
returnNone
result=self.func_call_reco()
returnASTNode(CommandNodeType,result)
deffunc_call_reco(self):
ifself.tokenType==WordTokType:
term=ASTNode(TermNodeType,self.token)
self.tokens.next()
ifself.tokenType==LParTokType:
self.tokens.next()
result=self.func_call_list_reco()
ifresult:
ifself.tokenType==RParTokType:
self.tokenType,self.token,self.lineNo=\
self.tokens.next()
returnASTNode(FuncCallNodeType,term,
result)
else:
raiseParseError(self.lineNo,'missingright
paren')
else:
raiseParseError(self.lineNo,'badfunccall
list')
else:
raiseParseError(self.lineNo,'missingleftparen')
else:
returnNone
deffunc_call_list_reco(self):
terms=[]
while1:
ifnotresult:
break
terms.append(result)
ifself.tokenType!=CommaTokType:
Page127
APythonBook
break
self.tokens.next()
returnASTNode(FuncCallListNodeType,terms)
#
#Theparseerrorexceptionclass.
#
classParseError(Exception):
def__init__(self,lineNo,msg):
RuntimeError.__init__(self,msg)
self.lineNo=lineNo
self.msg=msg
defgetLineNo(self):
returnself.lineNo
defgetMsg(self):
returnself.msg
defis_word(token):
forletterintoken:
ifletternotinstring.ascii_letters:
returnNone
return1
#
#Generatethetokens.
#Usage:
#gen=genTokens(infile)
#tokType,tok,lineNo=gen.next()
#...
defgenTokens(infile):
lineNo=0
while1:
lineNo+=1
try:
line=infile.next()
except:
yield(EOFTokType,None,lineNo)
toks=line.split()
fortokintoks:
ifis_word(tok):
tokType=WordTokType
eliftok=='(':
tokType=LParTokType
eliftok==')':
tokType=RParTokType
eliftok==',':
tokType=CommaTokType
yield(tokType,tok,lineNo)
deftest(infileName):
parser=ProgParser()
#ipshell('(test)#1\nCtrlDtoexit')
Page128
APythonBook
result=None
try:
result=parser.parseFile(infileName)
exceptParseError,exp:
sys.stderr.write('ParseError:(%d)%s\n'%\
(exp.getLineNo(),exp.getMsg()))
ifresult:
result.show(0)
defusage():
print__doc__
sys.exit(1)
defmain():
args=sys.argv[1:]
try:
except:
usage()
relink=1
foropt,valinopts:
usage()
iflen(args)!=1:
usage()
inputfile=args[0]
test(inputfile)
main()
Commentsandexplanation:
ThetokenizerisaPythongenerator.ItreturnsaPythongeneratorthatcan
produce"(tokType,tok,lineNo)"tuples.Ourtokenizerissosimplemindedthat
wehavetoseparateallofourtokenswithwhitespace.(Alittlelater,we'llseehow
tousePlextoovercomethislimitation.)
Theparserclass(ProgParser)containstherecognizermethodsthatimplementthe
productionrules.Eachofthesemethodsrecognizesasyntacticconstructdefined
byarule.Inourexample,thesemethodshavenamesthatendwith"_reco".
Wecouldhave,alternatively,implementedourrecognizersasglobalfunctions,
insteadofasmethodsinaclass.However,usingaclassgivesusaplaceto"hang"
thevariablesthatareneededacrossmethodsandsavesusfromhavingtouse
("evil")globalvariables.
Arecognizermethodrecognizesterminals(syntacticelementsontherighthand
sideofthegrammarruleforwhichthereisnogrammarrule)by(1)checkingthe
tokentypeandthetokenvalue,andthen(2)callingthetokenizertogetthenext
token(becauseithasconsumedatoken).
Page129
APythonBook
Arecognizermethodchecksforandprocessesanonterminal(syntacticelements
ontherighthandsideforwhichthereisagrammarrule)bycallingtherecognizer
methodthatimplementsthatnonterminal.
Ifarecognizermethodfindsasyntaxerror,itraisesanexceptionofclass
ParserError.
SinceourexamplerecursivedescentparsercreatesanAST(anabstractsyntax
tree),wheneverarecognizermethodsuccessfullyrecognizesasyntacticconstruct,
itcreatesaninstanceofclassASTNodetorepresentitandreturnsthatinstanceto
itscaller.TheinstanceofASTNodehasanodetypeandcontainschildnodes
whichwereconstructedbyrecognizermethodscalledbythisone(i.e.that
representnonterminalsontherighthandsideofagrammarrule).
Eachtimearecognizermethod"consumesatoken",itcallsthetokenizertoget
thenexttoken(andtokentypeandlinenumber).
Thetokenizerreturnsatokentypeinadditiontothetokenvalue.Italsoreturnsa
linenumberforerrorreporting.
ThesyntaxtreeisconstructedfrominstancesofclassASTNode.
TheASTNodeclasshasashowmethod,whichwalkstheASTandproduces
output.Youcanimaginethatasimilarmethodcoulddocodegeneration.And,
youshouldconsiderthepossibilityofwritinganalogoustreewalkmethodsthat
performtaskssuchasoptimization,annotationoftheAST,etc.
And,hereisasampleofthedatawecanapplythisparserto:
aaa()
bbb(ccc())
ddd(eee(),fff(ggg(),hhh(),iii()))
And,ifweruntheparseronthethisinputdata,wesee:
$pythonworkbook045.pyworkbook045.data
NodeTypeProgNodeType
NodeTypeCommandNodeType
NodeTypeFuncCallNodeType
NodeTypeTermNodeType
Child:aaa
NodeTypeFuncCallListNodeType
Child:bbb
Child:ccc
Page130
APythonBook
Child:ddd
Child:eee
Child:fff
Child:ggg
Child:hhh
Child:iii
2.6.3Creatingalexer/tokenizerwithPlex
LexicalanalysisThetokenizerinourrecursivedescentparserexamplewas(for
demonstrationpurposes)overlysimple.Youcanalwayswritemorecomplextokenizers
byhand.However,formorecomplex(andreal)tokenizers,youmaywanttouseatoolto
buildyourtokenizer.
Inthissectionwe'lldescribePlexanduseittoproduceatokenizerforourrecursive
descentparser.
YoucanobtainPlexathttp://www.cosc.canterbury.ac.nz/~greg/python/Plex/.
Inordertouseit,youmaywanttoaddPlex1.1.4/PlextoyourPYTHONPATH.
HereisasimpleexamplefromthePlextutorial:
"""
SamplePlexlexer
Usage:
pythonplex_example.pyinputfile
"""
importsys
importPlex
Page131
APythonBook
defcount_lines(scanner,text):
scanner.line_count+=1
print''*60
letter=Plex.Range("AZaz")
digit=Plex.Range("09")
name=letter+Plex.Rep(letter|digit)
number=Plex.Rep1(digit)
space=Plex.Any("\t")
endline=Plex.Str('\n')
#comment=Plex.Str('"')+Plex.Rep(Plex.AnyBut('"'))+
Plex.Str('"')
resword=Plex.Str("if","then","else","end")
lexicon=Plex.Lexicon([
(endline,count_lines),
(resword,'keyword'),
(name,'ident'),
(number,'int'),
(Plex.Any("+*/=<>"),'operator'),
(space,Plex.IGNORE),
#(comment,'comment'),
(Plex.Str('('),'lpar'),
(Plex.Str(')'),'rpar'),
#commentssurroundedby(*and*)
(Plex.Str("(*"),Plex.Begin('comment')),
Plex.State('comment',[
(Plex.Str("*)"),Plex.Begin('')),
(Plex.AnyChar,Plex.IGNORE),
]),
])
infile=open(infileName,"r")
scanner=Plex.Scanner(lexicon,infile,infileName)
scanner.line_count=0
whileTrue:
token=scanner.read()
iftoken[0]isNone:
break
position=scanner.position()
posstr=('(%d,%d)'%(position[1],
position[2],)).ljust(10)
tokstr='"%s"'%token[1]
tokstr=tokstr.ljust(20)
print'%stok:%stokType:%s'%(posstr,tokstr,token[0],)
print'line_count:%d'%scanner.line_count
defusage():
print__doc__
sys.exit(1)
defmain():
args=sys.argv[1:]
Page132
APythonBook
iflen(args)!=1:
usage()
infileName=args[0]
test(infileName)
main()
Hereisabitofdataonwhichwecanusetheabovelexer:
mass=(height*(*somecomment*)width*depth)/density
totalmass=totalmass+mass
And,whenweapplytheabovetestprogramtothisdata,hereiswhatwesee:
$pythonplex_example.pyplex_example.data
(1,0)tok:"mass"tokType:ident
(1,5)tok:"="tokType:operator
(1,7)tok:"("tokType:lpar
(1,8)tok:"height"tokType:ident
(1,15)tok:"*"tokType:operator
(1,36)tok:"width"tokType:ident
(1,42)tok:"*"tokType:operator
(1,44)tok:"depth"tokType:ident
(1,49)tok:")"tokType:rpar
(1,51)tok:"/"tokType:operator
(1,53)tok:"density"tokType:ident
(2,0)tok:"totalmass"tokType:ident
(2,10)tok:"="tokType:operator
(2,12)tok:"totalmass"tokType:ident
(2,22)tok:"+"tokType:operator
(2,24)tok:"mass"tokType:ident
line_count:2
Createalexiconfromscanningpatterns.
SeethePlextutorialandreference(andbelow)formoreinformationonhowto
constructthepatternsthatmatchvarioustokens.
Createascannerwithalexicon,aninputfile,andaninputfilename.
Thecall"scanner.read()"getsthenexttoken.Itreturnsatuplecontaining(1)the
tokenvalueand(2)thetokentype.
Thecall"scanner.position()"getsthepositionofthecurrenttoken.Itreturnsa
tuplecontaining(1)theinputfilename,(2)thelinenumber,and(3)thecolumn
number.
Wecanexecuteamethodwhenagiventokenisfoundbyspecifyingthefunction
asthetokenaction.Inourexample,thefunctioniscount_lines.Maintainingaline
Page133
APythonBook
countisactuallyunneeded,sincethepositiongivesusthisinformation.However,
noticehowweareabletomaintainavalue(inourcaseline_count)asan
attributeofthescanner.
And,herearesomecommentsonconstructingthepatternsusedinalexicon:
Plex.Rangeconstructsapatternthatmatchesanycharacterintherange.
Plex.Repconstructsapatternthatmatchesasequenceofzeroormoreitems.
Plex.Rep1constructsapatternthatmatchesasequenceofoneormoreitems.
pat1+pat2constructsapatternthatmatchesasequencecontainingpat1
followedbypat2.
pat1|pat2constructsapatternthatmatcheseitherpat1orpat2.
Plex.Anyconstructsapatternthatmatchesanyonecharacterinitsargument.
Nowlet'srevisitourrecursivedescentparser,thistimewithatokenizerbuiltwithPlex.
Thetokenizeristrivial,butwillserveasanexampleofhowtohookitintoaparser:
"""
ArecursivedescentparserexampleusingPlex.
ThisexampleusesPlextoimplementatokenizer.
Usage:
pythonpython_201_rparser_plex.py[options]<inputfile>
Options:
Example:
pythonpython_201_rparser_plex.pymyfile.txt
Thegrammar:
Command::=Func_call
Term=<word>
"""
importsys,string,types
importgetopt
importPlex
##fromIPython.ShellimportIPShellEmbed
##ipshell=IPShellEmbed((),
##banner='>>>>>>>>IntoIPython>>>>>>>>',
##exit_msg='<<<<<<<<OutofIPython<<<<<<<<')
#
#Constants
#
Page134
APythonBook
#ASTnodetypes
NoneNodeType=0
ProgNodeType=1
CommandNodeType=2
FuncCallNodeType=3
TermNodeType=5
#Tokentypes
NoneTokType=0
LParTokType=1
RParTokType=2
WordTokType=3
CommaTokType=4
EOFTokType=5
NodeTypeDict={
}
#
#
classASTNode:
self.children=[]
foriteminargs:
print'NodeType%s'%NodeTypeDict[self.nodeType]
level+=1
child.show(level)
foriteminchild:
item.show(level)
else:
print'Child:',child
print'',
Page135
APythonBook
#
#Therecursivedescentparserclass.
#Containsthe"recognizer"methods,whichimplementthegrammar
#rules(above),onerecognizermethodforeachproductionrule.
#
classProgParser:
def__init__(self):
self.tokens=None
self.token=''
self.lineNo=1
self.infile=None
self.tokens=None
defparseFile(self,infileName):
self.tokens=None
self.token=''
self.lineNo=1
self.infile=file(infileName,'r')
self.tokens=genTokens(self.infile,infileName)
try:
self.tokens.next()
self.infile.close()
self.infile=None
returnresult
defparseStream(self,instream):
self.tokens=None
self.token=''
self.lineNo=1
self.tokens=genTokens(self.instream,'<stream>')
try:
self.tokens.next()
raiseRuntimeError,'Emptystream'
self.infile.close()
self.infile=None
returnresult
defprog_reco(self):
commandList=[]
while1:
result=self.command_reco()
ifnotresult:
break
Page136
APythonBook
commandList.append(result)
returnASTNode(ProgNodeType,commandList)
defcommand_reco(self):
ifself.tokenType==EOFTokType:
returnNone
returnASTNode(CommandNodeType,result)
deffunc_call_reco(self):
ifself.tokenType==WordTokType:
term=ASTNode(TermNodeType,self.token)
self.tokens.next()
ifself.tokenType==LParTokType:
self.tokens.next()
result=self.func_call_list_reco()
ifresult:
ifself.tokenType==RParTokType:
self.tokenType,self.token,self.lineNo=\
self.tokens.next()
returnASTNode(FuncCallNodeType,term,
result)
else:
raiseParseError(self.lineNo,'missingright
paren')
else:
raiseParseError(self.lineNo,'badfunccall
list')
else:
raiseParseError(self.lineNo,'missingleftparen')
else:
returnNone
deffunc_call_list_reco(self):
terms=[]
while1:
ifnotresult:
break
terms.append(result)
ifself.tokenType!=CommaTokType:
break
self.tokens.next()
returnASTNode(FuncCallListNodeType,terms)
#
#Theparseerrorexceptionclass.
#
classParseError(Exception):
def__init__(self,lineNo,msg):
Page137
APythonBook
RuntimeError.__init__(self,msg)
self.lineNo=lineNo
self.msg=msg
defgetLineNo(self):
returnself.lineNo
defgetMsg(self):
returnself.msg
#
#Generatethetokens.
#Usageexample
#gen=genTokens(infile)
#tokType,tok,lineNo=gen.next()
#...
defgenTokens(infile,infileName):
letter=Plex.Range("AZaz")
digit=Plex.Range("09")
name=letter+Plex.Rep(letter|digit)
lpar=Plex.Str('(')
rpar=Plex.Str(')')
comma=Plex.Str(',')
comment=Plex.Str("#")+Plex.Rep(Plex.AnyBut("\n"))
space=Plex.Any("\t\n")
lexicon=Plex.Lexicon([
(name,'word'),
(lpar,'lpar'),
(rpar,'rpar'),
(comma,'comma'),
(comment,Plex.IGNORE),
(space,Plex.IGNORE),
])
scanner=Plex.Scanner(lexicon,infile,infileName)
while1:
tokenType,token=scanner.read()
name,lineNo,columnNo=scanner.position()
iftokenType==None:
tokType=EOFTokType
token=None
eliftokenType=='word':
tokType=WordTokType
eliftokenType=='lpar':
tokType=LParTokType
eliftokenType=='rpar':
tokType=RParTokType
eliftokenType=='comma':
tokType=CommaTokType
else:
tokType=NoneTokType
tok=token
yield(tokType,tok,lineNo)
parser=ProgParser()
Page138
APythonBook
#ipshell('(test)#1\nCtrlDtoexit')
result=None
try:
result=parser.parseFile(infileName)
exceptParseError,exp:
sys.stderr.write('ParseError:(%d)%s\n'%\
(exp.getLineNo(),exp.getMsg()))
ifresult:
result.show(0)
defusage():
print__doc__
sys.exit(1)
defmain():
args=sys.argv[1:]
try:
except:
usage()
foropt,valinopts:
usage()
iflen(args)!=1:
usage()
infileName=args[0]
test(infileName)
main()
And,hereisasampleofthedatawecanapplythisparserto:
#TestforrecursivedescentparserandPlex.
#Command#1
aaa()
#Command#2
bbb(ccc())#Anendoflinecomment.
#Command#3
#Endoftest
And,whenwerunourparser,itproducesthefollowing:
$pythonplex_recusive.pyplex_recusive.data
NodeTypeProgNodeType
Child:aaa
Page139
APythonBook
Child:bbb
Child:ccc
Child:ddd
Child:eee
Child:fff
Child:ggg
Child:hhh
Child:iii
Comments:
Wecannowputcommentsinourinput,andtheywillbeignored.Comments
beginwitha"#"andcontinuetotheendofline.Seethedefinitionofcommentin
functiongenTokens.
Thistokenizerdoesnotrequireustoseparatetokenswithwhitespaceasdidthe
simpletokenizerintheearlierversionofourrecursivedescentparser.
Thechangeswemadeovertheearlierversionwereto:
1. ImportPlex.
2. ReplacethedefinitionofthetokenizerfunctiongenTokens.
3. ChangethecalltogenTokenssothatthecallpassesinthefilename,whichis
neededtocreatethescanner.
OurnewversionofgenTokensdoesthefollowing:
1. Createpatternsforscanning.
2. Createalexicon(aninstanceofPlex.Lexicon),whichusesthepatterns.
Page140
APythonBook
3. Createascanner(aninstanceofPlex.Scanner),whichusesthelexicon.
4. Executealoopthatreadstokens(fromthescanner)and"yields"eachone.
2.6.4Asurveyofexistingtools
Forcomplexparsingtasks,youmaywanttoconsiderthefollowingtools:
kwParsingAparsergeneratorinPython
http://gadfly.sourceforge.net/kwParsing.html
PLYPythonLexYacchttp://systems.cs.uchicago.edu/ply/
PyLRFastLRparsinginpython
http://starship.python.net/crew/scott/PyLR.html
YappsTheYappsParserGeneratorSystem
http://theory.stanford.edu/~amitp/Yapps/
And,forlexicalanalysis,youmayalsowanttolookhere:
UsingRegularExpressionsforLexicalAnalysis
http://effbot.org/zone/xmlscanner.htm
Plexhttp://www.cosc.canterbury.ac.nz/~greg/python/Plex/.
Inthesectionsbelow,wegiveexamplesandnotesabouttheuseofPLYandpyparsing.
2.6.5CreatingaparserwithPLY
InthissectionwewillshowhowtoimplementourparserexamplewithPLY.
FirstdownloadPLY.Itisavailablehere:PLY(PythonLexYacc)
http://www.dabeaz.com/ply/
ThenaddthePLYdirectorytoyourPYTHONPATH.
LearnhowtoconstructlexersandparserswithPLYbyreadingdoc/ply.htmlinthe
distributionofPLYandbylookingattheexamplesinthedistribution.
Forthoseofyouwhowantamorecomplexexample,seeAPythonParserforthe
RELAXNGCompactSyntax,whichisimplementedwithPLY.
Now,hereisourexampleparser.Commentsandexplanationsarebelow:
"""
Aparserexample.
ThisexampleusesPLYtoimplementalexerandparser.
Thegrammar:
Prog::=Command*
Command::=Func_call
Page141
APythonBook
Func_call_list::=Func_call*
Term=<word>
Hereisasample"program"touseasinput:
#Command#1
aaa()
#Command#2
#Command#3
#Endoftest
"""
importsys
importtypes
importgetopt
importply.lexaslex
importply.yaccasyacc
#
#Globals
#
startlinepos=0
#
#Constants
#
#ASTnodetypes
NoneNodeType=0
ProgNodeType=1
CommandNodeType=2
CommandListNodeType=3
FuncCallNodeType=4
TermNodeType=6
NodeTypeDict={
CommandListNodeType:'CommandListNodeType',
}
#
Page142
APythonBook
#
classASTNode:
self.children=[]
foriteminargs:
defappend(self,item):
print'NodeType:%s'%NodeTypeDict[self.nodeType]
level+=1
child.show(level)
foriteminchild:
item.show(level)
else:
print'Value:',child
print'',
#
#Exceptionclasses
#
classLexerError(Exception):
def__init__(self,msg,lineno,columnno):
self.msg=msg
self.lineno=lineno
self.columnno=columnno
defshow(self):
sys.stderr.write('Lexererror(%d,%d)%s\n'%\
(self.lineno,self.columnno,self.msg))
classParserError(Exception):
def__init__(self,msg,lineno,columnno):
self.msg=msg
self.lineno=lineno
self.columnno=columnno
defshow(self):
sys.stderr.write('Parsererror(%d,%d)%s\n'%\
(self.lineno,self.columnno,self.msg))
#
#Lexerspecification
#
tokens=(
'NAME',
Page143
APythonBook
'LPAR','RPAR',
'COMMA',
)
#Tokens
t_LPAR=r'$'
t_RPAR=r'$'
t_COMMA=r'\,'
t_NAME=r'[azAZ_][azAZ09_]*'
#Ignorewhitespace
t_ignore='\t'
#Ignorecomments('#'toendofline)
deft_COMMENT(t):
r'\#[^\n]*'
pass
deft_newline(t):
r'\n+'
globalstartlinepos
startlinepos=t.lexer.lexpos1
t.lineno+=t.value.count("\n")
deft_error(t):
globalstartlinepos
msg="Illegalcharacter'%s'"%(t.value[0])
columnno=t.lexer.lexposstartlinepos
raiseLexerError(msg,t.lineno,columnno)
#
#Parserspecification
#
defp_prog(t):
'prog:command_list'
t[0]=ASTNode(ProgNodeType,t[1])
defp_command_list_1(t):
'command_list:command'
t[0]=ASTNode(CommandListNodeType,t[1])
defp_command_list_2(t):
'command_list:command_listcommand'
t[1].append(t[2])
t[0]=t[1]
defp_command(t):
'command:func_call'
t[0]=ASTNode(CommandNodeType,t[1])
defp_func_call_1(t):
'func_call:termLPARRPAR'
Page144
APythonBook
t[0]=ASTNode(FuncCallNodeType,t[1])
defp_func_call_2(t):
'func_call:termLPARfunc_call_listRPAR'
t[0]=ASTNode(FuncCallNodeType,t[1],t[3])
defp_func_call_list_1(t):
'func_call_list:func_call'
t[0]=ASTNode(FuncCallListNodeType,t[1])
defp_func_call_list_2(t):
'func_call_list:func_call_listCOMMAfunc_call'
t[1].append(t[3])
t[0]=t[1]
defp_term(t):
'term:NAME'
t[0]=ASTNode(TermNodeType,t[1])
defp_error(t):
globalstartlinepos
msg="Syntaxerrorat'%s'"%t.value
columnno=t.lexer.lexposstartlinepos
raiseParserError(msg,t.lineno,columnno)
#
#ParsetheinputanddisplaytheAST(abstractsyntaxtree)
#
defparse(infileName):
startlinepos=0
#Buildthelexer
lex.lex(debug=1)
#Buildtheparser
yacc.yacc()
#Readtheinput
infile=file(infileName,'r')
content=infile.read()
infile.close()
try:
#Dotheparse
result=yacc.parse(content)
#DisplaytheAST
result.show(0)
exceptLexerError,exp:
exp.show()
exceptParserError,exp:
exp.show()
USAGE_TEXT=__doc__
defusage():
printUSAGE_TEXT
sys.exit(1)
Page145
APythonBook
defmain():
args=sys.argv[1:]
try:
except:
usage()
relink=1
foropt,valinopts:
usage()
iflen(args)!=1:
usage()
infileName=args[0]
parse(infileName)
main()
Applyingthisparsertothefollowinginput:
#Command#1
aaa()
#Command#2
#Command#3
#Endoftest
producesthefollowingoutput:
NodeType:ProgNodeType
NodeType:CommandListNodeType
NodeType:CommandNodeType
NodeType:FuncCallNodeType
NodeType:TermNodeType
Value:aaa
Value:bbb
NodeType:FuncCallListNodeType
Value:ccc
Value:ddd
Page146
APythonBook
Value:eee
Value:fff
Value:ggg
Value:hhh
Value:iii
CreatingthesyntaxtreeBasically,eachrule(1)recognizesanonterminal,(2)
createsanode(possiblyusingthevaluesfromtherighthandsideoftherule),and
(3)returnsthenodebysettingthevalueoft[0].Adeviationfromthisisthe
processingofsequences,discussedbelow.
Sequencesp_command_list_1andp_command_list_1showhowtohandle
sequencesofitems.Inthiscase:
p_command_list_1recognizesacommandandcreatesaninstanceof
ASTNodewithtypeCommandListNodeTypeandaddsthecommandtoitasa
child,and
p_command_list_2recognizesanadditionalcommandandaddsit(asachild)
totheinstanceofASTNodethatrepresentsthelist.
DistinguishingbetweendifferentformsofthesameruleInordertoprocess
alternativestothesameproductionruledifferently,weusedifferentfunctions
withdifferentimplementations.Forexample,weuse:
p_func_call_1torecognizeandprocess"func_call:termLPARRPAR"(a
functioncallwithoutarguments),and
p_func_call_2torecognizeandprocess"func_call:termLPARfunc_call_list
RPAR"(afunctioncallwitharguments).
ReportingerrorsOurparserreportsthefirsterrorandquits.We'vedonethisby
raisinganexceptionwhenwefindanerror.Weimplementtwoexceptionclasses:
LexerErrorandParserError.Implementingmorethanoneexceptionclassenables
ustodistinguishbetweendifferentclassesoferrors(notethemultipleexcept:
clausesonthetry:statementinfunctionparse).And,weuseaninstanceofthe
exceptionclassasacontainerinorderto"bubbleup"informationabouttheerror
(e.g.amessage,alinenumber,andacolumnnumber).
Page147
APythonBook
2.6.6Creatingaparserwithpyparsing
pyparsingisarelativelynewparsingpackageforPython.Itwasimplementedandis
supportedbyPaulMcGuireanditshowspromise.Itappearsespeciallyeasytouseand
seemsespeciallyappropriateinparticularforquickparsingtasks,althoughithasfeatures
thatmakesomecomplexparsingtaskseasy.ItfollowsaverynaturalPythonstylefor
constructingparsers.
Gooddocumentationcomeswiththepyparsingdistribution.Seefile
HowToUseParsing.html.So,Iwon'ttrytorepeatthathere.Whatfollowsisanattemptto
provideseveralquickexamplestohelpyousolvesimpleparsingtasksasquicklyas
possible.
Youwillalsowanttolookatthesamplesintheexamplesdirectory,whicharevery
helpful.Myexamplesbelowarefairlysimple.Youcanseemoreoftheabilityof
pyparsingtohandlecomplextasksintheexamples.
WheretogetitYoucanfindpyparsingat:PyparsingWikiHome
http://pyparsing.wikispaces.com/
HowtoinstallitPutthepyparsingmodulesomewhereonyourPYTHONPATH.
Andnow,hereareafewexamples.
2.6.6.1Parsingcommadelimitedlines
Note:Thisexampleisfordemonstrationpurposesonly.Ifyoureallytoneedtoparse
commadelimitedfields,youcanprobablydosomuchmoreeasilywiththeCSV(comma
separatedvalues)moduleinthePythonstandardlibrary.
Hereisasimplegrammarforlinescontainingfieldsseparatedbycommas:
importsys
frompyparsingimportalphanums,ZeroOrMore,Word
fieldDef=Word(alphanums)
lineDef=fieldDef+ZeroOrMore(","+fieldDef)
deftest():
args=sys.argv[1:]
iflen(args)!=1:
print'usage:pythonpyparsing_test1.py<datafile.txt>'
sys.exit(1)
infilename=sys.argv[1]
infile=file(infilename,'r')
forlineininfile:
fields=lineDef.parseString(line)
printfields
Page148
APythonBook
test()
Hereissomesampledata:
abcd,defg
11111,22222,33333
And,whenwerunourparseronthisdatafile,hereiswhatwesee:
$pythoncomma_parser.pysample1.data
['abcd',',','defg']
['11111',',','22222',',','33333']
NotehowthegrammarisconstructedfromnormalPythoncallstofunctionand
object/classconstructors.I'veconstructedtheparserinlinebecausemyexample
issimple,butconstructingtheparserinafunctionorevenamodulemightmake
senseformorecomplexgrammars.pyparsingmakesiteasytousethesethese
differentstyles.
Use"+"tospecifyasequence.Inourexample,alineDefisafieldDef
followedby....
UseZeroOrMoretospecifyrepetition.Inourexample,alineDefisa
fieldDeffollowedbyzeroormoreoccurancesofcommaandfieldDef.
ThereisalsoOneOrMorewhenyouwanttorequireatleastoneoccurance.
Parsingcommadelimitedtexthappenssofrequentlythatpyparsingprovidesa
shortcut.Replace:
lineDef=fieldDef+ZeroOrMore(","+fieldDef)
with:
lineDef=delimitedList(fieldDef)
AndnotethatdelimitedListtakesanoptionalargumentdelimusedtospecify
thedelimiter.Thedefaultisacomma.
2.6.6.2Parsingfunctors
Thisexampleparsesexpressionsoftheformfunc(arg1,arg2,arg3):
frompyparsingimportWord,alphas,alphanums,nums,ZeroOrMore,
Literal
lparen=Literal("(")
rparen=Literal(")")
identifier=Word(alphas,alphanums+"_")
integer=Word(nums)
functor=identifier
arg=identifier|integer
Page149
APythonBook
args=arg+ZeroOrMore(","+arg)
expression=functor+lparen+args+rparen
deftest():
content=raw_input("Enteranexpression:")
parsedContent=expression.parseString(content)
printparsedContent
test()
Explanation:
UseLiteraltospecifyafixedstringthatistobematchedexactly.Inourexample,
alparenisa(.
Wordtakesanoptionalsecondargument.Withasingle(string)argument,it
matchesanycontiguouswordmadeupofcharactersinthestring.Withtwo
(string)argumentsitmatchesawordwhosefirstcharacterisinthefirststringand
whoseremainingcharactersareinthesecondstring.So,ourdefinitionof
identifiermatchesawordwhosefirstcharacterisanalphaandwhoseremaining
charactersarealphanumericsorunderscore.Asanotherexample,youcanthink
ofWord("0123456789")asanalogoustoaregexpcontainingthepattern"[09]+".
Useaverticalbarforalternation.Inourexample,anargcanbeeitheranidentifier
oraninteger.
2.6.6.3Parsingnames,phonenumbers,etc.
Thisexampleparsesexpressionshavingthefollowingform:
Inputformat:
[name][phone][city,statezip]
Last,first1112223333city,ca99999
Hereistheparser:
importsys
frompyparsingimportalphas,nums,ZeroOrMore,Word,Group,
Suppress,Combine
lastname=Word(alphas)
firstname=Word(alphas)
city=Group(Word(alphas)+ZeroOrMore(Word(alphas)))
state=Word(alphas,exact=2)
zip=Word(nums,exact=5)
name=Group(lastname+Suppress(",")+firstname)
phone=Combine(Word(nums,exact=3)+""+Word(nums,exact=3)+""
+Word(nums,exact=4))
location=Group(city+Suppress(",")+state+zip)
record=name+phone+location
Page150
APythonBook
deftest():
args=sys.argv[1:]
iflen(args)!=1:
print'usage:pythonpyparsing_test3.py<datafile.txt>'
sys.exit(1)
infilename=sys.argv[1]
infile=file(infilename,'r')
forlineininfile:
line=line.strip()
iflineandline[0]!="#":
fields=record.parseString(line)
printfields
test()
And,hereissomesampleinput:
Jabberer,Jerry1112223333Bakersfield,CA95111
Kackler,Kerry1112223334Fresno,CA95112
Louderdale,Larry1112223335LosAngeles,CA94001
Hereisoutputfromparsingtheaboveinput:
[['Jabberer','Jerry'],'1112223333',[['Bakersfield'],'CA',
'95111']]
[['Kackler','Kerry'],'1112223334',[['Fresno'],'CA','95112']]
[['Louderdale','Larry'],'1112223335',[['Los','Angeles'],'CA',
'94001']]
Comments:
Weusethelen=nargumenttotheWordconstructortoresticttheparserto
acceptingaspecificnumberofcharacters,forexampleinthezipcodeandphone
number.Wordalsoacceptsmin=n''and``max=ntoenableyoutorestrict
thelengthofawordtowithinarange.
WeuseGrouptogrouptheparsedresultsintosublists,forexampleinthe
definitionofcityandname.Groupenablesustoorganizetheparseresultsinto
simpleparsetrees.
WeuseCombinetojoinparsedresultsbackintoasinglestring.Forexample,in
thephonenumber,wecanrequiredashesandyetjointheresultsbackintoa
singlestring.
WeuseSuppresstoremoveunneededsubelementsfromparsedresults.For
example,wedonotneedthecommabetweenlastandfirstname.
2.6.6.4Amorecomplexexample
Thisexample(thankstoPaulMcGuire)parsesamorecomplexstructureandproducesa
dictionary.
Page151
APythonBook
Hereisthecode:
frompyparsingimportLiteral,Word,Group,Dict,ZeroOrMore,alphas,
nums,\
delimitedList
importpprint
testData="""
++++++++++
||A1|B1|C1|D1|A2|B2|C2|D2|
+=======+======+======+======+======+======+======+======+======+
|min|7|43|7|15|82|98|1|37|
|max|11|52|10|17|85|112|4|39|
|ave|9|47|8|16|84|106|3|38|
|sdev|1|3|1|1|1|3|1|1|
++++++++++
"""
#Definegrammarfordatatable
heading=(Literal(
"++++++++++")
+
"||A1|B1|C1|D1|A2|B2|C2|D2|"+
"+=======+======+======+======+======+======+======+======+======+").
suppress()
vert=Literal("|").suppress()
number=Word(nums)
rowData=Group(vert+Word(alphas)+vert+
delimitedList(number,"|")+
vert)
trailing=Literal(
"++++++++++").
suppress()
datatable=heading+Dict(ZeroOrMore(rowData))+trailing
defmain():
#Nowparsedataandprintresults
data=datatable.parseString(testData)
print"data:",data
print"data.asList():",
pprint.pprint(data.asList())
print"datakeys:",data.keys()
print"data['min']:",data['min']
print"data.max:",data.max
main()
Whenwerunthis,itproducesthefollowing:
data:[['min','7','43','7','15','82','98','1','37'],
Page152
APythonBook
['max','11','52','10','17','85','112','4','39'],
['ave','9','47','8','16','84','106','3','38'],
['sdev','1','3','1','1','1','3','1','1']]
data.asList():[['min','7','43','7','15','82','98','1','37'],
['max','11','52','10','17','85','112','4','39'],
['ave','9','47','8','16','84','106','3','38'],
['sdev','1','3','1','1','1','3','1','1']]
datakeys:['ave','min','sdev','max']
data['min']:['7','43','7','15','82','98','1','37']
data.max:['11','52','10','17','85','112','4','39']
Notes:
NotetheuseofDicttocreateadictionary.Theprintstatementsshowhowtoget
attheitemsinthedictionary.
NotehowwecanalsogettheparseresultsasalistbyusingmethodasList.
Again,weusesuppresstoremoveunneededitemsfromtheparseresults.
2.7GUIApplications
2.7.1Introduction
ThissectionwillhelpyoutoputaGUI(graphicaluserinterface)inyourPython
program.
WewilluseaparticularGUIlibrary:PyGTK.We'vechosenthisbecauseitisreasonably
lightweightandourgoalistoembedlightweightGUIinterfacesinan(possibly)
existingapplication.
ForsimplerGUIneeds,considerEasyGUI,whichisalsodescribedbelow.
FormoreheavyweightGUIneeds(forexample,completeGUIapplications),youmay
wanttoexploreWxPython.SeetheWxPythonhomepageat:http://www.wxpython.org/
2.7.2PyGtk
InformationaboutPyGTKishere:ThePyGTKhomepagehttp://www.pygtk.org//.
2.7.2.1Asimplemessagedialogbox
InthissectionweexplainhowtopopupasimpledialogboxfromyourPython
application.
Todothis,dothefollowing:
1. ImportgtkintoyourPythonmodule.
2. Definethedialoganditsbehavior.
Page153
APythonBook
3. Createaninstanceofthedialog.
4. Runtheeventloop.
Hereisasamplethatdisplaysamessagebox:
importsys
importgetopt
importgtk
classMessageBox(gtk.Dialog):
def__init__(self,message="",buttons=(),pixmap=None,
modal=True):
gtk.Dialog.__init__(self)
self.connect("destroy",self.quit)
self.connect("delete_event",self.quit)
ifmodal:
self.set_modal(True)
hbox=gtk.HBox(spacing=5)
hbox.set_border_width(5)
self.vbox.pack_start(hbox)
hbox.show()
ifpixmap:
self.realize()
pixmap=Pixmap(self,pixmap)
hbox.pack_start(pixmap,expand=False)
pixmap.show()
label=gtk.Label(message)
hbox.pack_start(label)
label.show()
fortextinbuttons:
b=gtk.Button(text)
b.set_flags(gtk.CAN_DEFAULT)
b.set_data("user_data",text)
b.connect("clicked",self.click)
self.action_area.pack_start(b)
b.show()
self.ret=None
defquit(self,*args):
self.hide()
self.destroy()
gtk.main_quit()
defclick(self,button):
self.ret=button.get_data("user_data")
self.quit()
#createamessagebox,andreturnwhichbuttonwaspressed
defmessage_box(title="MessageBox",message="",buttons=(),
pixmap=None,
modal=True):
win=MessageBox(message,buttons,pixmap=pixmap,modal=modal)
win.set_title(title)
Page154
APythonBook
win.show()
gtk.main()
returnwin.ret
deftest():
result=message_box(title='Test#1',
message='Hereisyourmessage',
buttons=('Ok','Cancel'))
print'result:',result
USAGE_TEXT="""
Usage:
pythonsimple_dialog.py[options]
Options:
Example:
pythonsimple_dialog.py
"""
defusage():
printUSAGE_TEXT
sys.exit(1)
defmain():
args=sys.argv[1:]
try:
except:
usage()
relink=1
foropt,valinopts:
usage()
iflen(args)!=0:
usage()
test()
main()
Someexplanation:
First,weimportgtk
NextwedefineaclassMessageBoxthatimplementsamessagebox.Herearea
fewimportantthingstoknowaboutthatclass:
Itisasubclassofgtk.Dialog.
Itcreatesalabelandpacksitintothedialog'sclientarea.NotethataDialogis
aWindowthatcontainsavboxatthetopofandanaction_areaatthebottom
ofitsclientarea.Theintensionisforustopackmiscellaneouswidgetsinto
thevboxandtoputbuttonssuchas"Ok","Cancel",etcintotheaction_area.
Page155
APythonBook
Itcreatesonebuttonforeachbuttonlabelpassedtoitsconstructor.The
buttonsareallconnectedtotheclickmethod.
Theclickmethodsavesthevalueoftheuser_dataforthebuttonthatwas
clicked.Inourexample,thisvaluewillbeeither"Ok"or"Cancel".
And,wedefineafunction(message_box)that(1)createsaninstanceofthe
MessageBoxclass,(2)setsitstitle,(3)showsit,(4)startsitseventloopsothatit
cangetandprocesseventsfromtheuser,and(5)returnstheresulttothecaller(in
thiscase"Ok"or"Cancel").
Ourtestingfunction(test)callsfunctionmessage_boxandprintstheresult.
Thislookslikequiteabitofcode,untilyounoticethattheclassMessageBoxand
thefunctionmessage_boxcouldbeputitautilitymoduleandreused.
2.7.2.2Asimpletextinputdialogbox
And,hereisanexamplethatdisplaysantextinputdialog:
importsys
importgetopt
importgtk
classEntryDialog(gtk.Dialog):
def__init__(self,message="",default_text='',modal=True):
gtk.Dialog.__init__(self)
ifmodal:
box=gtk.VBox(spacing=10)
box.set_border_width(10)
self.vbox.pack_start(box)
box.show()
ifmessage:
label=gtk.Label(message)
box.pack_start(label)
label.show()
self.entry=gtk.Entry()
self.entry.set_text(default_text)
box.pack_start(self.entry)
self.entry.show()
self.entry.grab_focus()
button=gtk.Button("OK")
button.connect("clicked",self.click)
button.set_flags(gtk.CAN_DEFAULT)
self.action_area.pack_start(button)
button.show()
button.grab_default()
button=gtk.Button("Cancel")
Page156
APythonBook
button.connect("clicked",self.quit)
button.set_flags(gtk.CAN_DEFAULT)
self.action_area.pack_start(button)
button.show()
self.ret=None
defquit(self,w=None,event=None):
self.hide()
self.destroy()
gtk.main_quit()
defclick(self,button):
self.ret=self.entry.get_text()
self.quit()
definput_box(title="InputBox",message="",default_text='',
modal=True):
win=EntryDialog(message,default_text,modal=modal)
win.show()
gtk.main()
returnwin.ret
deftest():
result=input_box(title='Test#2',
message='Enteravaluexxx:',
default_text='adefaultvalue')
ifresultisNone:
print'Canceled'
else:
print'result:"%s"'%result
USAGE_TEXT="""
Usage:
Options:
Example:
"""
defusage():
printUSAGE_TEXT
sys.exit(1)
defmain():
args=sys.argv[1:]
try:
except:
usage()
relink=1
foropt,valinopts:
usage()
Page157
APythonBook
iflen(args)!=0:
usage()
test()
main()
Mostoftheexplanationforthemessageboxexampleisrelevanttothisexample,too.
Herearesomedifferences:
OurEntryDialogclassconstructorcreatesinstanceofgtk.Entry,setsitsdefault
value,andpacksitintotheclientarea.
Theconstructoralsoautomaticallycreatestwobuttons:"OK"and"Cancel".The
"OK"buttonisconnecttotheclickmethod,whichsavesthevalueoftheentry
field.The"Cancel"buttonisconnecttothequitmethod,whichdoesnotsavethe
value.
And,ifclassEntryDialogandfunctioninput_boxlookusableanduseful,add
themtoyourutilityguimodule.
2.7.2.3Afileselectiondialogbox
Thisexampleshowsafileselectiondialogbox:
importsys
importgetopt
importgtk
classFileChooser(gtk.FileSelection):
def__init__(self,modal=True,multiple=True):
gtk.FileSelection.__init__(self)
self.multiple=multiple
ifmodal:
self.cancel_button.connect('clicked',self.quit)
self.ok_button.connect('clicked',self.ok_cb)
ifmultiple:
self.set_select_multiple(True)
self.ret=None
defquit(self,*args):
self.hide()
self.destroy()
gtk.main_quit()
defok_cb(self,b):
ifself.multiple:
self.ret=self.get_selections()
Page158
APythonBook
else:
self.ret=self.get_filename()
self.quit()
deffile_sel_box(title="Browse",modal=False,multiple=True):
win=FileChooser(modal=modal,multiple=multiple)
win.show()
gtk.main()
returnwin.ret
deffile_open_box(modal=True):
returnfile_sel_box("Open",modal=modal,multiple=True)
deffile_save_box(modal=True):
returnfile_sel_box("SaveAs",modal=modal,multiple=False)
deftest():
result=file_open_box()
print'openresult:',result
result=file_save_box()
print'saveresult:',result
USAGE_TEXT="""
Usage:
Options:
Example:
"""
defusage():
printUSAGE_TEXT
sys.exit(1)
defmain():
args=sys.argv[1:]
try:
except:
usage()
relink=1
foropt,valinopts:
usage()
iflen(args)!=0:
usage()
test()
main()
#importpdb
#pdb.run('main()')
Page159
APythonBook
Alittleguidance:
Thereisapredefinedfileselectiondialog.Wesubclassit.
Thisexampledisplaysthefileselectiondialogtwice:oncewithatitle"Open"and
oncewithatitle"SaveAs".
Notehowwecancontrolwhetherthedialogallowsmultiplefileselections.And,
ifweselectthemultipleselectionmode,thenweuseget_selectionsinsteadof
get_filenameinordertogettheselectedfilenames.
Thedialogcontainsbuttonsthatenabletheuserto(1)createanewfolder,(2)
deleteafile,and(3)renameafile.Ifyoudonotwanttheusertoperformthese
operations,thencallhide_fileop_buttons.Thiscalliscommentedoutinour
samplecode.
Notethattherearealsopredefineddialogsforfontselection(FontSelectionDialog)and
colorselection(ColorSelectionDialog)
2.7.3EasyGUI
IfyourGUIneedsareminimalist(maybeapopupdialogortwo)andyourapplicationis
imperativeratherthaneventdriven,thenyoumaywanttoconsiderEasyGUI.Asthe
namesuggests,itisextremelyeasytouse.
HowtoknowwhenyoumightbeabletouseEasyGUI:
Yourapplicationdoesnotneedtoruninawindowcontainingmenusandamenu
bar.
YourGUIneedsamounttolittlemorethandisplayingadialognowandthento
getresponsesfromtheuser.
Youdonotwanttowriteaneventdrivenapplication,thatis,oneinwhichyour
codesitsandwaitsforthetheusertoinitiateoperation,forexample,withmenu
items.
EasyGUIplusdocumentationandexamplesareavailableatEasyGUIhomepageat
SourceForgehttp://easygui.sourceforge.net/
EasyGUIprovidesfunctionsforavarietyofcommonlyneededdialogboxes,including:
Amessageboxdisplaysamessage.
Ayes/nomessageboxdisplays"Yes"and"No"buttons.
Acontinue/cancelmessageboxdisplays"Continue"and"Cancel"buttons.
Achoiceboxdisplaysaselectionlist.
Anenterboxallowsentryofalineoftext.
Anintegerboxallowsentryofaninterger.
Amultipleentryboxallowsentryintomultiplefields.
Codeandtextboxessupportthedisplayoftextinmonospacedorporportional
Page160
APythonBook
fonts.
Fileanddirectoryboxesenabletheusertoselectafileoradirectory.
SeethedocumentationattheEasyGUIWebsiteformorefeatures.
ForademonstrationofEasyGUI'scapabilities,runtheeasygui.pyasaPythonscript:
$pythoneasygui.py
2.7.3.1AsimpleEasyGUIexample
Hereisasimpleexamplethatpromptstheuserforanentry,thenshowstheresponseina
messagebox:
importeasygui
deftesteasygui():
response=easygui.enterbox(msg='Enteryourname:',title='Name
Entry')
easygui.msgbox(msg=response,title='YourResponse')
testeasygui()
2.7.3.2AnEasyGUIfileopendialogexample
Thisexamplepresentsadialogtoallowtheusertoselectafile:
importeasygui
deftest():
response=easygui.fileopenbox(msg='Selectafile')
print'filename:%s'%response
test()
2.8GuidanceonPackagesandModules
2.8.1Introduction
Pythonhasanexcellentrangeofimplementationorganizationstructures.Theserange
fromstatementsandcontrolstructures(atalowlevel)throughfunctions,methods,and
classes(atanintermediatelevel)andmodulesandpackagesatanupperlevel.
Thissectionprovidessomeguidancewiththeuseofpackages.Inparticular:
Howtoconstructandimplementthem.
Howtousethem.
Howtodistributeandinstallthem.
Page161
APythonBook
2.8.2ImplementingPackages
APythonpackageisacollectionofPythonmodulesinadiskdirectory.
Inordertobeabletoimportindividualmodulesfromadirectory,thedirectorymust
containafilenamed__init__.py.(Notethatrequirementdoesnotapplytodirectoriesthat
arelistedinPYTHONPATH.)The__init__.pyservesseveralpurposes:
Thepresenceofthefile__init__.pyinadirectorymarksthedirectoryasaPython
package,whichenablesimportingmodulesfromthedirectory.
Thefirsttimeanapplicationimportsanymodulefromthedirectory/package,the
codeinthemodule__init__isevaluated.
Ifthepackageitselfisimported(asopposedtoanindividualmodulewithinthe
directory/package),thenitisthe__init__thatisimported(andevaluated).
2.8.3UsingPackages
Onesimplewaytoenabletheusertoimportanduseapackageistoinstructtheuseto
importindividualmodulesfromthepackage.
Asecond,slightlymoreadvancedwaytoenabletheusertoimportthepackageisto
exposethosefeaturesofthepackageinthe__init__module.Supposethatmodulemod1
containsfunctionsfun1aandfun1bandsupposethatmodulemod2containsfunctions
fun2aandfun2b.Thenfile__init__.pymightcontainthefollowing:
frommod1importfun1a,fun1b
frommod2importfun2a,fun2b
Then,ifthefollowingisevaluatedintheuser'scode:
importtestpackages
Thentestpackageswillcontainfun1a,fun1b,fun2a,andfun2b.
Forexample,hereisaninteractivesessionthatdemostratesimportingthepackage:
>>>importtestpackages
>>>printdir(testpackages)
[`__builtins__',`__doc__',`__file__',`__name__',
`__path__',
`fun1a',`fun1b',`fun2a',`fun2b',`mod1',`mod2']
2.8.4DistributingandInstallingPackages
Distutils(PythonDistributionUtilities)hasspecialsupportfordistrubutingandinstalling
packages.Learnmorehere:DistributingPythonModules
http://docs.python.org/distutils/index.html.
Page162
APythonBook
Asourexample,imaginethatwehaveadirectorycontainingthefollowing:
Testpackages
Testpackages/README
Testpackages/MANIFEST.in
Testpackages/setup.py
Testpackages/testpackages/__init__.py
Testpackages/testpackages/mod1.py
Testpackages/testpackages/mod2.py
NoticethesubdirectoryTestpackages/testpackagescontainingthefile__init__.py.
ThisisthePythonpackagethatwewillinstall.
We'lldescribehowtoconfiguretheabovefilessothattheycanbepackagedasasingle
distributionfileandsothatthePythonpackagetheycontaincanbeinstalledasapackage
byDistutils.
TheMANIFEST.infileliststhefilesthatwewantincludedinourdistribution.Hereis
thecontentsofourMANIFEST.infile:
includeREADMEMANIFESTMANIFEST.in
includesetup.py
includetestpackages/*.py
Thesetup.pyfiledescribestoDistutils(1)howtopackagethedistributionfileand(2)
howtoinstallthedistribution.Hereisthecontentsofoursamplesetup.py:
fromdistutils.coreimportsetup#[1]
long_description='TestsforinstallinganddistributingPython
packages'
setup(name='testpackages',#[2]
version='1.0a',
description='TestsforPythonpackages',
maintainer='DaveKuhlman',
maintainer_email='dkuhlman@rexx.com',
url='http://www.rexx.com/~dkuhlman',
long_description=long_description,
packages=['testpackages']#[3]
)
Explanation:
1. WeimportthenecessarycomponentfromDistutils.
2. Wedescribethepackageanditsdeveloper/maintainer.
3. Wespecifythedirectorythatistobeinstalledasapackage.Whentheuser
installsourdistribution,thisdirectoryandallthemodulesinitwillbeinstalledas
apackage.
Page163
APythonBook
Now,tocreateadistributionfile,werunthefollowing:
pythonsetup.pysdistformats=gztar
whichwillcreateafiletestpackages1.0a.tar.gzunderthedirectorydist.
Then,youcangivethisdistributionfiletoapotentialuser,whocaninstallitbydoingthe
following:
$tarxvzftestpackages1.0a.tar.gz
$cdtestpackages1.0a
$pythonsetup.pybuild
$pythonsetup.pyinstall#asroot
2.9EndMatter
2.9.1AcknowledgementsandThanks
ThankstotheimplementorsofPythonforproducinganexceptionallyusableand
enjoyableprogramminglanguage.
ThankstoDaveBeazleyandothersforSWIGandPLY.
ThankstoGregEwingforPyrexandPlex.
ThankstoJamesHenstridgeforPyGTK.
2.9.2SeeAlso
ThemainPythonWebSitehttp://www.python.orgformoreinformationon
Python.
PythonDocumentationhttp://www.python.org/doc/forlotsofdocumentation
onPython
Dave'sWebSitehttp://http://www.davekuhlman.orgformoresoftwareand
informationonusingPythonforXMLandtheWeb.
TheSWIGhomepagehttp://www.swig.orgformoreinformationonSWIG
(SimplifiedWrapperandInterfaceGenerator).
ThePyrexhomepagehttp://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/
formoreinformationonPyrex.
PLY(PythonLexYacc)homepagehttp://www.dabeaz.com/ply/formore
informationonPLY.
ThePlexhomepagehttp://www.cosc.canterbury.ac.nz/greg.ewing/python/Plex/
formoreinformationonPlex.
DistributingPythonModuleshttp://docs.python.org/distutils/index.htmlfor
informationonthePythonDistributionUtilities(Distutils).
Page164
APythonBook
3Part3PythonWorkbook
3.1Introduction
ThisdocumenttakesaworkbookandexercisewithsolutionsapproachtoPython
training.Itishopedthatthosewhofeelaneedforlessexplanationandmorepractical
exerciseswillfindthisuseful.
Afewnotesabouttheexercises:
I'vetriedtoincludesolutionsformostoftheexercises.Hopefully,youwillbe
abletocopyandpastethesesolutionsintoyourtexteditor,thenextendand
experimentwiththem.
IusetwointeractivePythoninterpreters(althoughtheyarethesamePython
underneath).Whenyouseethisprompt>>>,it'sthestandardPythoninterpreter.
And,whenyouseethispromptIn[1]:,it'sIPython
http://ipython.scipy.org/moin/.
ThelatestversionofthisdocumentisatmyWebsite(URLabove).
Ifyouhavecommentsorsuggestions,pleasesendthemmyway.
3.2LexicalStructures
3.2.1Variablesandnames
Anameisanycombinationofletters,digits,andtheunderscore,butthefirstcharacter
mustbealetteroranunderscore.Namesmaybeofanylength.
Caseissignificant.
Exercises:
1. Whichofthefollowingarevalidnames?
1. total
2. total_of_all_vegetables
3. bigtitle1
4. _inner_func
5. 1bigtitle
6. bigtitle1
2. Whichorthefollowingpairsarethesamename:
1. the_last_itemandthe_last_item
Page165
APythonBook
2. the_last_itemandThe_Last_Item
3. itemianditemj
4. item1anditeml
Solutions:
1. Items1,2,4,and6arevalid.Item3isnotasinglename,butisthreeitems
separatedbytheminusoperator.Item5isnotvalidbecauseitbeginswithadigit.
2. Pythonnamesarecasesensitive,whichmeans:
1. the_last_itemandthe_last_itemarethesame.
2. the_last_itemandThe_Last_ItemaredifferentThesecondname
hasanuppercasecharacters.
3. itemianditemjaredifferent.
4. item1anditemlaredifferentThisonemaybedifficulttosee,
dependingonthefontyouareviewing.Onenameendswiththedigitone;the
otherendswiththealphacharacter"el".Andthisexampleprovidesagood
reasontouse"1"and"l"judiciouslyinnames.
ThefollowingarekeywordsinPythonandshouldnotbeusedasvariablenames:
anddelfromnotwhile
aselifglobalorwith
assertelseifpassyield
breakexceptimportprint
classexecinraise
continuefinallyisreturn
defforlambdatry
Exercises:
1. WhichofthefollowingarevalidnamesinPython?
1. _global
2. global
3. file
Solutions:
1. Donotusekeywordsforvariablenames:
1. Valid
2. Notavalidname."global"isakeyword.
3. Valid,however,"file"isthenameofabuiltintype,asyouwilllearnlater,so
youareadvisednottoredefineit.Hereareafewofthenamesofbuiltin
types:"file","int","str","float","list","dict",etc.SeeBuiltinTypes
http://docs.python.org/lib/types.htmlformorebuiltintypes..
ThefollowingareoperatorsinPythonandwillseparatenames:
+***///%
<<>>&|^~
Page166
APythonBook
<><=>===!=<>
andorisnotin
Also:()[].(dot)
But,notethatthePythonstyleguidesuggeststhatyouplaceblanksaroundbinary
operators.Oneexceptiontothisruleisfunctionargumentsandparametersforfunctions:
itissuggestedthatyounotputblanksaroundtheequalsign(=)usedtospecifykeyword
argumentsanddefaultparameters.
Exercises:
1. Whichofthefollowingaresinglenamesandwhicharenamesseparatedby
operators?
1. fruit_collection
2. fruitcollection
Solutions:
1. Donotuseadash,orotheroperator,inthemiddleofaname:
1. fruit_collectionisasinglename
2. fruitcollectionistwonamesseparatedbyadash.
3.2.2Linestructure
InPython,normallywewriteonestatementperline.Infact,Pythonassumesthis.
Therefore:
Statementseparatorsarenotnormallyneeded.
But,ifwewantmorethanonestatementonaline,weuseastatementseparator,
specificallyasemicolon.
And,ifwewanttoextendastatementtoasecondorthirdlineandsoon,we
sometimesneedtodoabitextra.
ExtendingaPythonstatementtoasubsequentlineFollowthesetworules:
1. Ifthereisanopencontext,nothingspecialneedbedonetoextendastatement
acrossmultiplelines.Anopencontextisanopenparenthesis,anopensquare
bracket,oranopencurlybracket.
2. Wecanalwaysextendastatementonafollowinglinebyplacingabackslashas
thelastcharacteroftheline.
Exercises:
1. Extendthefollowingstatementtoasecondlineusingparentheses:
total_count=tree_count+vegetable_count+
fruit_count
2. Extendthefollowingstatementtoasecondlineusingthebackslashline
Page167
APythonBook
continuationcharacter:
total_count=tree_count+vegetable_count+
fruit_count
Solutions:
1. ParenthesescreateanopencontextthattellsPythonthatastatementextendsto
thenextline:
total_count=(tree_count+
vegetable_count+fruit_count)
2. AbackslashasthelastcharacteronlinetellsPythonthatthecurrentstatement
extendstothenextline:
total_count=tree_count+\
vegetable_count+fruit_count
Forextendingalineonasubsequentline,whichisbetter,parenthesesorabackslash?
Hereisaquote:
"ThepreferredwayofwrappinglonglinesisbyusingPython'simplied
linecontinuationinsideparentheses,bracketsandbraces.Ifnecessary,
youcanaddanextrapairofparenthesesaroundanexpression,but
sometimesusingabackslashlooksbetter."
PEP8:StyleGuideforPythonCode
3.2.3Indentationandprogramstructure
Pythonusesindentationtoindicateprogramstructure.Thatistosay,inordertonesta
blockofcodeinsideacompoundstatement,youindentthatnestedcode.Thisisdifferent
frommanyprogramminglanguageswhichusesomesortofbeginandendmarkers,for
examplecurlybrackets.
ThestandardcodingpracticeforPythonistousefourspacesperindentationlevelandto
notusehardtabs.(SeetheStyleGuideforPythonCode.)Becauseofthis,youwillwant
touseatexteditorthatyoucanconfiguresothatitwillusefourspacesforindentation.
SeehereforalistofPythonfriendlytexteditors:PythonEditors.
Exercises:
1. Giventhefollowing,nesttheprintstatementinsidetheifstatement:
ifx>0:
printx
2. Nestthesetwolines:
Page168
APythonBook
z=x+y
printz
insidethefollowingfunctiondefinitionstatement:
defshow_sum(x,y):
Solutions:
1. Indentationindicatesthatonestatementisnestedinsideanotherstatement:
ifx>0:
printx
2. Indentationindicatesthatablockofstatementsisnestedinsideanotherstatement:
defshow_sum(x,y):
z=x+y
printz
3.3ExecutionModel
Hereareafewrules:
1. PythonevaluatesPythoncodefromthetopofamoduledowntothebottomofa
module.
2. Bindingstatementsattoplevelcreatenames(andbindvaluestothosenames)as
Pythonevaluatescode.Furthermore,anameisnotcreateduntilitisboundtoa
value/object.
3. Anestedreferencetoaname(forexample,insideafunctiondefinitionorinthe
nestedblockofanifstatement)isnotuseduntilthatnestedcodeisevaluated.
Exercises:
1. Willthefollowingcodeproduceanerror?
show_version()
defshow_version():
print'Version1.0a'
2. Willthefollowingcodeproduceanerror?
deftest():
show_version()
defshow_version():
print'Version1.0a'
test()
3. Willthefollowingcodeproduceanerror?Assumethatshow_configisnot
defined:
x=3
Page169
APythonBook
ifx>5:
show_config()
Solutions:
1. Answer:Yes,itgeneratesanerror.Thenameshow_versionwouldnotbe
createdandboundtoavalueuntilthedeffunctiondefinitionstatementbindsa
functionobjecttoit.Thatisdoneaftertheattempttouse(call)thatobject.
2. Answer:No.Thefunctiontest()doescallthefunctionshow_version(),
butsincetest()isnotcalleduntilaftershow_version()isdefined,thatis
OK.
3. Answer:No.It'sbadcode,butinthiscasewillnotgenerateanerror.Sincexis
lessthan5,thebodyoftheifstatementisnotevaluated.
N.B.Thisexampleshowswhyitisimportantduringtestingthateverylineof
codeinyourPythonprogrambeevaluated.HereisgoodPythonicadvice:"Ifit's
nottested,it'sbroken."
3.4BuiltinDataTypes
Eachofthesubsectionsinthissectiononbuiltindatatypeswillhaveasimilarstructure:
1. Abriefdescriptionofthedatatypeanditsuses.
2. RepresentationandconstructionHowtorepresentaninstanceofthedatatype.
Howtocodealiteralrepresentationthatcreatesanddefinesaninstance.Howto
createaninstanceofthebuiltintype.
3. Operatorsthatareapplicabletothedatatype.
4. Methodsimplementedandsupportedbythedatatype.
3.4.1Numbers
Thenumbersyouwillusemostcommonlyarelikelytobeintegersandfloats.Python
alsohaslongintegersandcomplexnumbers.
Afewfactsaboutnumbers(inPython):
Pythonwillconverttousingalongintegerautomaticallywhenneeded.Youdo
notneedtoworryaboutexceedingthesizeofa(standard)integer.
ThesizeofthelargestintegerinyourversionofPythonisinsys.maxint.To
learnwhatitis,do:
>>>importsys
>>>printsys.maxint
9223372036854775807
Theaboveshowthemaximumsizeofanintegerona64bitversionofPython.
Youcanconvertfromintegertofloatbyusingthefloatconstructor.Example:
Page170
APythonBook
>>>x=25
>>>y=float(x)
>>>printy
25.0
Pythondoes"mixedarithmetic".Youcanadd,multiply,anddivideintegersand
floats.Whenyoudo,Python"promotes"theresulttoafloat.
3.4.1.1Literalrepresentationsofnumbers
Anintegerisconstructedwithaseriesofdigitsortheintegerconstructor(int(x)).Be
awarethatasequenceofdigitsbeginningwithzerorepresentsanoctalvalue.Examples:
>>>x1=1234
>>>x2=int('1234')
>>>x3=25
>>>x1
1234
>>>x2
1234
>>>x3
25
Afloatisconstructedeitherwithdigitsandadot(example,12.345)orwith
engineering/scientificnotationorwiththefloatconstructor(float(x)).Examples:
>>>x1=2.0e3
>>>x1=1.234
>>>x2=1.234
>>>x3=float('1.234')
>>>x4=2.0e3
>>>x5=2.0e3
>>>printx1,x2,x3,x4,x5
1.2341.2341.2342000.00.002
Exercises:
Constructthesenumericvalues:
1.
2.
3.
4.
5.
6.
7.
Integerzero
Floatingpointzero
Integeronehundredandone
Floatingpointonethousand
Floatingpointonethousandusingscientificnotation
Createapositiveinteger,anegativeinteger,andzero.Assignthemtovariables
Writeseveralarithmeticexpressions.Bindthevaluestovariables.Useavariety
ofoperators,e.g.+,,/,*,etc.Useparenthesestocontroloperatorscope.
8. Createseveralfloatsandassignthemtovariables.
9. Writeseveralarithmeticexpressionscontainingyourfloatvariables.
Page171
APythonBook
10. Writeseveralexpressionsusingmixedarithmetic(integersandfloats).Obtaina
floatasaresultofdivisionofoneintegerbyanother;dosobyexplicitly
convertingoneintegertoafloat.
Solutions:
1.
2.
3.
4.
5.
6.
0
0.0,0.,or.0
101
1000.0
1e3or1.0e3
Asigningintegervaluestovariables:
In[7]:value1=23
In[8]:value2=14
In[9]:value3=0
In[10]:value1
Out[10]:23
In[11]:value2
Out[11]:14
In[12]:value3
Out[12]:0
7. Assigningexpressionvaluestovariables:
value1=4*(3+5)
value2=(value1/3.0)2
8. Assigningfloatstovariables:
value1=0.01
value2=3.0
value3=3e4
9. Assigningexpressionscontainingvarialbes:
value4=value1*(value2value3)
value4=value1+value2+value3value4
10. Mixedarithmetic:
x=5
y=8
z=float(x)/y
Youcanalsoconstructintegersandfloatsusingtheclass.Callingaclass(using
parenthesesafteraclassname,forexample)producesaninstanceoftheclass.
Exercises:
1. Constructanintegerfromthestring"123".
2. Constructafloatfromtheinteger123.
3. Constructanintegerfromthefloat12.345.
Solutions:
Page172
APythonBook
1. Usetheintdatatypetoconstructanintegerinstancefromastring:
int("123")
2. Usethefloatdatatypetoconstructafloatinstancefromaninteger:
float(123)
3. Usetheintdatatypetoconstructanintegerinstancefromafloat:
int(12.345)#>12
Noticethattheresultistruncatedtotheintegerpart.
3.4.1.2Operatorsfornumbers
Youcanusemostofthefamiliaroperatorswithnumbers,forexample:
+***///%
<<>>&|^~
<><=>===!=<>
Lookhereforanexplanationoftheseoperatorswhenappliedtonumbers:Numeric
Typesint,float,long,complexhttp://docs.python.org/lib/typesnumeric.html.
Someoperatorstakeprecedenceoverothers.ThetableintheWebpagejustreferenced
abovealsoshowsthatorderofpriority.
Hereisabitofthattable:
Allnumerictypes(exceptcomplex)supportthefollowingoperations,
sortedbyascendingpriority(operationsinthesameboxhavethe
same
priority;allnumericoperationshaveahigherprioritythan
comparison
operations):
OperationResult
x+ysumofxandy
xydifferenceofxandy
x*yproductofxandy
x/yquotientofxandy
x//y(floored)quotientofxandy
x%yremainderofx/y
xxnegated
+xxunchanged
abs(x)absolutevalueormagnitudeofx
int(x)xconvertedtointeger
long(x)xconvertedtolonginteger
float(x)xconvertedtofloatingpoint
complex(re,im)acomplexnumberwithrealpartre,imaginarypart
im.imdefaultstozero.
c.conjugate()conjugateofthecomplexnumberc
Page173
APythonBook
divmod(x,y)thepair(x//y,x%y)
pow(x,y)xtothepowery
x**yxtothepowery
Noticealsothatthesameoperatormayperformadifferentfunctiondependingonthe
datatypeofthevaluetowhichitisapplied.
Exercises:
1. Addthenumbers3,4,and5.
2. Add2totheresultofmultiplying3by4.
3. Add2plus3andmultiplytheresultby4.
Solutions:
1. Arithmeticexpressionsarefollowstandardinfixalgebraicsyntax:
3+4+5
2. Useanotherinfixexpression:
2+3*4
Or:
2+(3*4)
But,inthiscasetheparenthesesarenotnecessarybecausethe*operatorbinds
moretightlythanthe+operator.
3. Useparenthesestocontrolorderofevaluation:
(2+3)*4
Notethatthe*operatorhasprecedenceover(bindstighterthan)the+operator,
sotheparenthesesareneeded.
Pythondoesmixedarithemetic.Whenyouapplyanoperationtoanintegerandafloat,it
promotestheresulttothe"higher"datatype,afloat.
Ifyouneedtoperformanoperationonseveralintegers,butwantuseafloatingpoint
operation,firstconvertoneoftheintegerstoafloatusingfloat(x),whicheffectively
createsaninstanceofclassfloat.
TrythefollowingatyourPythoninteractiveprompt:
1. 1.0+2
2. 2/3Noticethattheresultistruncated.
3. float(2)/3Noticethattheresultisnottruncated.
Exercises:
1. Giventhefollowingassignments:
x=20
y=50
Page174
APythonBook
Dividexbyygivingafloatresult.
Solutions:
1. Promoteoneoftheintegerstofloatbeforeperformingthedivision:
z=float(x)/y
3.4.1.3Methodsonnumbers
Mostofthemethodsimplementedbythedatatypes(classes)intandfloatarespecial
methodsthatarecalledthroughtheuseofoperators.Specialmethodsoftenhavenames
thatbeginandendwithadoubleunderscore.Toseealistofthespecialnamesandabit
ofanindicationofwheneachiscalled,doanyofthefollowingatthePythoninteractive
prompt:
>>>help(int)
>>>help(32)
>>>help(float)
>>>help(1.23)
>>>dir(1)
>>>dir(1.2)
3.4.2Lists
Listsareacontainerdatatypethatactsasadynamicarray.Thatistosay,alistisa
sequencethatcanbeindexedintoandthatcangrowandshrink.
Atupleisanindexablecontainer,likealist,exceptthatatupleisimmutable.
Afewcharacteristicsoflistsandtuples:
Alisthasa(current)lengthGetthelengthofalistwithlen(mylist).
AlisthasanorderTheitemsinalistareordered,andyoucanthinkofthat
orderasgoingfromlefttoright.
AlistisheterogeneousYoucaninsertdifferenttypesofobjectsintothesame
list.
Listsaremutable,buttuplesarenot.Thus,thefollowingaretrueoflists,butnot
oftuples:
Youcanextendedoraddtoalist.
Youcanshrinkalistbydeletingitemsfromit.
Youcaninsertitemsintothemiddleofalistoratthebeginningofalist.You
canadditemstotheendofalist.
Youcanchangewhichitemisatagivenpositioninalist.
Page175
APythonBook
3.4.2.1Literalrepresentationoflists
Theliteralrepresentationofalistissquarebracketscontainingzeroormoreitems
separatedbycommas.
Examples:
1. TrytheseatthePythoninteractiveprompt:
>>>[11,22,33]
>>>['aa','bb','cc',]
>>>[100,'apple',200,'banana',]#Thelastcomma
is
>>>optional.
2. Alistcancontainlists.Infactalistcancontainanykindofobject:
>>>[1,[2,3],4,[5,6,7,],8]
3. Listsareheterogenous,thatis,differentkindsofobjectscanbeinthesamelist.
Hereisalistthatcontainsanumber,astring,andanotherlist:
>>>[123,'abc',[456,789]]
Exercises:
1. Create(define)thefollowingtuplesandlistsusingaliteral:
1. Atupleofintegers
2. Atupleofstrings
3. Alistofintegers
4. Alistofstrings
5. Alistoftuplesortupleoflists
6. Alistofintegersandstringsandtuples
7. Atuplecontainingexactlyoneitem
8. Anemptytuple
2. Doeachofthefollowing:
1. Printthelengthofalist.
2. PrinteachiteminthelistIterateovertheitemsinoneofyourlists.Print
eachitem.
3. Appendanitemtoalist.
4. Insertanitematthebeginningofalist.Insertaniteminthemiddleofalist.
5. Addtwoliststogether.Dosobyusingboththeextendmethodandtheplus
(+)operator.Whatisthedifferencebetweenextendingalistandaddingtwo
lists?
6. Retrievethe2nditemfromoneofyourtuplesorlists.
7. Retrievethe2nd,3rd,and4thitems(aslice)fromoneofyourtuplesorlists.
8. Retrievethelast(rightmost)iteminoneofyourlists.
9. Replaceaniteminalistwithanewitem.
Page176
APythonBook
10. Poponeitemofftheendofyourlist.
11. Deleteanitemfromalist.
12. Dothefollowinglistmanipulations:
1. Writeafunctionthattakestwoarguments,alistandanitem,andthat
appendstheitemtothelist.
2. Createanemptylist,
3. Callyourfunctionseveraltimestoappenditemstothelist.
4. Then,printouteachiteminthelist.
Solutions:
1. WecandefinelistliteralsatthePythonorIPythoninteractiveprompt:
1. Createatupleusingcommas,optionallywithparentheses:
In[1]:a1=(11,22,33,)
In[2]:a1
Out[2]:(11,22,33)
2. Quotedcharactersseparatedbycommascreateatupleofstrings:
In[3]:a2=('aaa','bbb','ccc')
In[4]:a2
Out[4]:('aaa','bbb','ccc')
3. Itemsseparatedbycommasinsidesquarebracketscreatealist:
In[26]:a3=[100,200,300,]
In[27]:a3
Out[27]:[100,200,300]
4. Stringsseparatedbycommasinsidesquarebracketscreatealistofstrings:
In[5]:a3=['basil','parsley','coriander']
In[6]:a3
Out[6]:['basil','parsley','coriander']
In[7]:
5. Atupleoralistcancontaintuplesandlists:
In[8]:a5=[(11,22),(33,44),(55,)]
In[9]:a5
Out[9]:[(11,22),(33,44),(55,)]
6. Alistortuplecancontainitemsofdifferenttypes:
In[10]:a6=[101,102,'abc',"def",(201,202),
('ghi','jkl')]
In[11]:a6
Out[11]:[101,102,'abc','def',(201,202),
('ghi','jkl')]
7. Inordertocreateatuplecontainingexactlyoneitem,wemustuseacomma:
In[13]:a7=(6,)
In[14]:a7
Page177
APythonBook
Out[14]:(6,)
8. Inordertocreateanemptytuple,usethetupleclass/typetocreateaninstance
ofaemptytuple:
In[21]:a=tuple()
In[22]:a
Out[22]:()
In[23]:type(a)
Out[23]:<type'tuple'>
3.4.2.2Operatorsonlists
Thereareseveraloperatorsthatareapplicabletolists.Hereishowtofindoutabout
them:
Dodir([])ordir(any_list_instance).Someoftheitemswith
specialnames(leadingandtrainingdoubleunderscores)willgiveyoucluesabout
operatorsimplementedbythelisttype.
Dohelp([])orhelp(list)atthePythoninteractiveprompt.
Dohelp(any_list_instance.some_method),wheresome_method
isoneoftheitemslistedusingdir(any_list_instance).
SeeSequenceTypesstr,unicode,list,tuple,buffer,xrange
http://docs.python.org/lib/typesseq.html
Exercises:
1. Concatenate(add)twoliststogether.
2. Createasinglelistthatcontainstheitemsinaninitiallistrepeated3times.
3. Comparetwolists.
Solutions:
1. Theplusoperator,appliedtotwolistsproducesanewlistthatisaconcatenation
oftwolists:
>>>[11,22]+['aa','bb']
2. Multiplyingalistbyanintegerncreatesanewlistthatrepeatstheoriginallistn
times:
>>>[11,'abc',4.5]*3
3. Thecomparisonoperatorscanbeusedtocomparelists:
>>>[11,22]==[11,22]
>>>[11,22]<[11,33]
3.4.2.3Methodsonlists
Again,usedir()andhelp()tolearnaboutthemethodssupportedbylists.
Page178
APythonBook
Examples:
1. Createtwo(small)lists.Extendthefirstlistwiththeitemsinthesecond.
2. Appendseveralindividualitemstotheendofalist.
3. (a)Insertaitematthebeginningofalist.(b)Insertanitemsomewhereinthe
middleofalist.
4. Popanitemofftheendofalist.
Solutions:
1. Theextendmethodaddselementsfromanotherlist,orotheriterable:
>>>a=[11,22,33,44,]
>>>b=[55,66]
>>>a.extend(b)
>>>a
[11,22,33,44,55,66]
2. Usetheappendmethodonalisttoadd/appendanitemtotheendofalist:
>>>a=['aa',11]
>>>a.append('bb')
>>>a.append(22)
>>>a
['aa',11,'bb',22]
3. Theinsertmethodonalistenablesustoinsertitemsatagivenpositionina
list:
>>>a=[11,22,33,44,]
>>>a.insert(0,'aa')
>>>a
['aa',11,22,33,44]
>>>a.insert(2,'bb')
>>>a
['aa',11,'bb',22,33,44]
But,notethatweuseappendtoadditemsattheendofalist.
4. Thepopmethodonalistreturnsthe"rightmost"itemfromalistandremoves
thatitemfromthelist:
>>>a=[11,22,33,44,]
>>>
>>>b=a.pop()
>>>a
[11,22,33]
>>>b
44
>>>b=a.pop()
>>>a
[11,22]
>>>b
33
Page179
APythonBook
Notethattheappendandpopmethodstakentogethercanbeusedtoimplement
astack,thatisaLIFO(lastinfirstout)datastructure.
3.4.2.4Listcomprehensions
Alistcomprehensionisaconvenientwaytoproducealistfromaniterable(asequence
orotherobjectthatcanbeiteratedover).
Initssimplestform,alistcomprehensionresemblestheheaderlineofaforstatement
insidesquarebrackets.However,inalistcomprehension,theforstatementheaderis
prefixedwithanexpressionandsurroundedbysquarebrackets.Hereisatemplate:
[expr(x)forxiniterable]
where:
expr(x)isanexpression,usually,butnotalways,containingx.
iterableissomeiterable.Aniterablemaybeasequence(forexample,alist,a
string,atuple)oranunorderedcollectionoraniterator(somethingoverwhichwe
caniterateorapplyaforstatementto).
Hereisanexample:
>>>a=[11,22,33,44]
>>>b=[x*2forxina]
>>>b
[22,44,66,88]
Exercises:
1. Giventhefollowinglistofstrings:
names=['alice','bertrand','charlene']
producethefollowinglists:(1)alistofalluppercasenames;(2)alistof
capitalized(firstletteruppercase);
2. Giventhefollowingfunctionwhichcalculatesthefactorialofanumber:
deft(n):
ifn<=1:
returnn
else:
returnn*t(n1)
andthefollowinglistofnumbers:
numbers=[2,3,4,5]
createalistofthefactorialsofeachofthenumbersinthelist.
Solutions:
1. Forourexpressioninalistcomprehension,usetheupperandcapitalize
Page180
APythonBook
methods:
>>>names=['alice','bertrand','charlene']
>>>[name.upper()fornameinnames]
['ALICE','BERTRAND','CHARLENE']
>>>[name.capitalize()fornameinnames]
['Alice','Bertrand','Charlene']
2. Theexpressioninourlistcomprehensioncallsthefactorialfunction:
deft(n):
ifn<=1:
returnn
else:
returnn*t(n1)
deftest():
numbers=[2,3,4,5]
factorials=[t(n)forninnumbers]
print'factorials:',factorials
test()
Alistcomprehensioncanalsocontainanifclause.Hereisatemplate:
[expr(x)forxiniterableifpred(x)]
where:
pred(x)isanexpressionthatevaluatestoatrue/falsevalue.Valuesthatcount
asfalsearenumericzero,False,None,andanyemptycollection.Allother
valuescountastrue.
Onlyvaluesforwhichtheifclauseevaluatestotrueareincludedincreatingtheresulting
list.
Examples:
>>>a=[11,22,33,44]
>>>b=[x*3forxinaifx%2==0]
>>>b
[66,132]
Exercises:
1. Giventwolists,generatealistofallthestringsinthefirstlistthatarenotinthe
secondlist.Herearetwosamplelists:
names1=['alice','bertrand','charlene','daniel']
names2=['bertrand','charlene']
Solutions:
1. Theifclauseofourlistcomprehensionchecksforcontainmentinthelistnames2:
Page181
APythonBook
deftest():
names1=['alice','bertrand','charlene',
'daniel']
names2=['bertrand','charlene']
names3=[namefornameinnames1ifnamenotin
names2]
print'names3:',names3
test()
Whenrun,thisscriptprintsoutthefollowing:
names3:['alice','daniel']
3.4.3Strings
Astringisanorderedsequenceofcharacters.Hereareafewcharacteristicsofstrings:
Astringhasalength.Getthelengthwiththelen()builtinfunction.
Astringisindexable.Getasinglecharacteratapositioninastringwiththe
squarebracketoperator,forexamplemystring[5].
Youcanretrieveaslice(substring)ofastringwithasliceoperation,forexample
mystring[5:8].
Createstringswithsinglequotesordoublequotes.Youcanputsinglequotesinside
doublequotesandyoucanputdoublequotesinsidesinglequotes.Youcanalsoescape
characterswithabackslash.
Exercises:
1. Createastringcontainingasinglequote.
2. Createastringcontainingadoublequote.
3. Createastringcontainingbothasinglequoteadoublequote.
Solutions:
1. Createastringwithdoublequotestoincludesinglequotesinsidethestring:
>>>str1="thatisjerry'sball"
2. Createastringenclosedwithsinglequotesinordertoincludedoublequotes
insidethestring:
>>>str1='say"goodbye",bullwinkle'
3. Takeyourchoice.Escapeeitherthesinglequotesorthedoublequoteswitha
backslash:
>>>str1='say"hello"tojerry\'smom'
>>>str2="say\"hello\"tojerry'smom"
>>>str1
'say"hello"tojerry\'smom'
Page182
APythonBook
>>>str2
'say"hello"tojerry\'smom'
Triplequotesenableyoutocreateastringthatspansmultiplelines.Usethreesingle
quotesorthreedoublequotestocreateasinglequotedstring.
Examples:
1. Createatriplequotedstringthatcontainssingleanddoublequotes.
Solutions:
1. Usetriplesinglequotesortripledoublequotestocreatemultilinestrings:
String1='''Thisstringextends
acrossseverallines.And,soithas
endoflinecharactersinit.
'''
String2="""
Thisstringbeginsandendswithanendofline
character.Itcanhaveboth'single'
quotesand"double"quotesinit.
"""
deftest():
printString1
printString2
test()
3.4.3.1Characters
Pythondoesnothaveadistinctcharactertype.InPython,acharacterisastringoflength
1.Youcanusetheord()andchr()builtinfunctionstoconvertfromcharacterto
integerandback.
Exercises:
1. Createacharacter"a".
2. Createacharacter,thenobtainitsintegerrepresentation.
Solutions:
1. Thecharacter"a"isaplainstringoflength1:
>>>x='a'
2. Theintegerequivalentoftheletter"A":
>>>x="A"
>>>ord(x)
65
Page183
APythonBook
3.4.3.2Operatorsonstrings
Youcanconcatenatestringswiththe"+"operator.
Youcancreatemultipleconcatenatedcopiesofastringwiththe"*"operator.
And,augmentedassignment(+=and*=)alsowork.
Examples:
>>>'cat'+'and'+'dog'
'catanddog'
>>>'#'*40
'########################################'
>>>
>>>s1='flower'
>>>s1+='s'
>>>s1
'flowers'
Exercises:
1. Giventhesestrings:
>>>s1='abcd'
>>>s2='efgh'
createanewstringcomposedofthefirststringfollowedby(concatenatedwith)
thesecond.
2. Createasinglestringcontaining5copiesofthestring'abc'.
3. Usethemultiplicationoperatortocreatea"line"of50dashes.
4. Herearethecomponentsofapathtoafileonthefilesystem:"home",
"myusername","Workdir","notes.txt".Concatenatethesetogetherseparating
themwiththepathseparatortoformacompletepathtothatfile.(Notethatifyou
usethebackslashtoseparatecomponentsofthepath,youwillneedtousea
doublebackslash,becausethebackslashistheescapecharacterinstrings.
Solutions:
1. Theplus(+)operatorappliedtoastringcanbeusedtoconcatenatestrings:
>>>s3=s1+s2
>>>s3
'abcdefgh'
2. Themultiplicationoperator(*)appliedtoastringcreatesanewstringthat
concatenatesastringwithitselfsomenumberoftimes:
>>>s1='abc'*5
>>>s1
'abcabcabcabcabc'
3. Themultiplicationoperator(*)appliedtoastringcanbeusedtocreatea
Page184
APythonBook
"horizontaldividerline":
>>>s1=''*50
>>>prints1
4. Thesepmemberoftheosmodulegivesusaplatformindependentwayto
constructpaths:
>>>importos
>>>
>>>a=["home","myusername","Workdir","notes.txt"]
>>>path=a[0]+os.sep+a[1]+os.sep+a[2]+
os.sep+a[3]
>>>path
'home/myusername/Workdir/notes.txt'
And,amoreconcisesolution:
>>>importos
>>>a=["home","myusername","Workdir","notes.txt"]
>>>os.sep.join(a)
'home/myusername/Workdir/notes.txt'
Notes:
Notethatimportingtheosmoduleandthenusingos.sepfromthatmodule
givesusaplatformindependentsolution.
Ifyoudodecidetocodethepathseparatorcharacterexplicitlyandifyouare
onMSWindowswherethepathseparatoristhebackslash,thenyouwillneed
touseadoublebackslash,becausethatcharacteristheescapecharacter.
3.4.3.3Methodsonstrings
Stringsupportavarietyofoperations.Youcanobtainalistofthesemethodsbyusingthe
dir()builtinfunctiononanystring:
>>>dir("")
['__add__','__class__','__contains__','__delattr__','__doc__',
'__eq__','__ge__','__getattribute__','__getitem__',
'__getnewargs__','__getslice__','__gt__','__hash__','__init__',
'__le__','__len__','__lt__','__mod__','__mul__','__ne__',
'__new__','__reduce__','__reduce_ex__','__repr__','__rmod__',
'__rmul__','__setattr__','__str__','capitalize','center',
'count','decode','encode','endswith','expandtabs','find',
'index','isalnum','isalpha','isdigit','islower','isspace',
'istitle','isupper','join','ljust','lower','lstrip',
'partition','replace','rfind','rindex','rjust','rpartition',
'rsplit','rstrip','split','splitlines','startswith','strip',
'swapcase','title','translate','upper','zfill']
And,youcangethelponanyspecificmethodbyusingthehelp()builtinfunction.
Hereisanexample:
Page185
APythonBook
>>>help("".strip)
Helponbuiltinfunctionstrip:
strip(...)
S.strip([chars])>stringorunicode
ReturnacopyofthestringSwithleadingandtrailing
whitespaceremoved.
IfcharsisgivenandnotNone,removecharactersinchars
instead.
Ifcharsisunicode,Swillbeconvertedtounicodebefore
stripping
Exercises:
1. Stripallthewhitespacecharactersofftherightendofastring.
2. Centerashortstringwithinalongerstring,thatis,padashortstringwithblank
charactersonbothrightandlefttocenterit.
3. Convertastringtoalluppercase.
4. Splitastringintoalistof"words".
5. (a)Jointhestringsinalistofstringstoformasinglestring.(b)Ditto,butputa
newlinecharacterbetweeneachoriginalstring.
Solutions:
1. Therstrip()methodstripswhitespaceofftherightsideofastring:
>>>s1='sometext\n'
>>>s1
'sometext\n'
>>>s2=s1.rstrip()
>>>s2
'sometext'
2. Thecenter(n)methodcentersastringwithinapaddedstringofwidthn:
>>>s1='Dave'
>>>s2=s1.center(20)
>>>s2
'Dave'
3. Theupper()methodproducesanewstringthatconvertsallalphacharactersin
theoriginaltouppercase:
>>>s1='Banana'
>>>s1
'Banana'
>>>s2=s1.upper()
>>>s2
'BANANA'
4. Thesplit(sep)methodproducesalistofstringsthatareseparatedbysepin
theoriginalstring.Ifsepisomitted,whitespaceistreatedastheseparator:
Page186
APythonBook
>>>s1="""howdoesitfeel
...tobeonyourown
...nodirectionsknown
...likearollingstone
..."""
>>>words=s1.split()
>>>words
['how','does','it','feel','to','be','on','your',
'own','no',
'directions','known','like','a','rolling','stone']
Notethatthesplit()functioninthere(regularexpression)moduleisuseful
whentheseparatorismorecomplexthanwhitespaceorasinglecharacter.
5. Thejoin()methodconcatenatesstringsfromalistofstringstoformasingle
string:
>>>lines=[]
>>>lines.append('howdoesitfeel')
>>>lines.append('tobeonyourown')
>>>lines.append('nodirectionsknown')
>>>lines.append('likearollingstone')
>>>lines
['howdoesitfeel','tobeonyourown','no
directionsknown',
'likearollingstone']
>>>s1=''.join(lines)
>>>s2=''.join(lines)
>>>s3='\n'.join(lines)
>>>s1
'howdoesitfeeltobeonyourownnodirections
knownlikearollingstone'
>>>s2
'howdoesitfeeltobeonyourownnodirectionsknown
likearollingstone'
>>>s3
'howdoesitfeel\ntobeonyourown\nnodirections
known\nlikearollingstone'
>>>prints3
howdoesitfeel
tobeonyourown
nodirectionsknown
likearollingstone
3.4.3.4Rawstrings
Rawstringsgiveusaconvenientwaytoincludethebackslashcharacterinastring
withoutescaping(withanadditionalbackslash).Rawstringslooklikeplainliteral
strings,butareprefixedwithan"r"or"R".SeeStringliterals
http://docs.python.org/reference/lexical_analysis.html#stringliterals
Excercises:
Page187
APythonBook
1. Createastringthatcontainsabackslashcharacterusingbothplainliteralstring
andarawstring.
Solutions:
1. Weusean"r"prefixtodefinearawstring:
>>>print'abc\\def'
abc\def
>>>printr'abc\def'
abc\def
3.4.3.5Unicodestrings
Unicodestringsgiveusaconsistentwaytoprocesscharacterdatafromavarietyof
characterencodings.
Excercises:
1. Createseveralunicodestrings.Useboththeunicodeprefixcharacter("u")andthe
unicodetype(unicode(some_string)).
2. Convertastring(possiblyfromanothernonasciiencoding)tounicode.
3. Convertaunicodestringtoanotherencoding,forexample,utf8.
4. Testastringtodetermineifitisunicode.
5. Createastringthatcontainsaunicodecharacter,thatis,acharacteroutsidethe
asciicharacterset.
Solutions:
1. Wecanrepresentunicodestringwitheitherthe"u"prefixorwithacalltothe
unicodetype:
defexercise1():
a=u'abcd'
printa
b=unicode('efgh')
printb
2. Weconvertastringfromanothercharacterencodingintounicodewiththe
decode()stringmethod:
importsys
defexercise2():
a='abcd'.decode('utf8')
printa
b='abcd'.decode(sys.getdefaultencoding())
printb
3. Wecanconvertaunicodestringtoanothercharacterencodingwiththe
encode()stringmethod:
importsys
Page188
APythonBook
defexercise3():
a=u'abcd'
printa.encode('utf8')
printa.encode(sys.getdefaultencoding())
4. Herearetwowaystocheckthetypeofastring:
importtypes
defexercise4():
a=u'abcd'
printtype(a)istypes.UnicodeType
printtype(a)istype(u'')
5. Wecanencodeunicodecharactersinastringinseveralways,forexample,(1)by
definingautf8stringandconvertingittounicodeor(2)definingastringwithan
embeddedunicodecharacteror(3)concatenatingaunicodecharacherintoa
string:
defexercise5():
utf8_string='IvanKrsti\xc4\x87'
unicode_string=utf8_string.decode('utf8')
printunicode_string.encode('utf8')
printlen(utf8_string)
printlen(unicode_string)
unicode_string=u'aa\u0107bb'
unicode_string='aa'+unichr(263)+'bb'
Guidanceforuseofencodingsandunicode:
1. Convert/decodefromanexternalencodingtounicodeearly:
my_source_string.decode(encoding)
2. Doyourwork(Pythonprocessing)inunicode.
3. Convert/encodetoanexternalencodinglate(forexample,justbeforesavingtoan
externalfile):
my_unicode_string.encode(encoding)
Formoreinformation,see:
UnicodeInPython,CompletelyDemystifiedhttp://farmdev.com/talks/unicode/
UnicodeHowtohttp://www.amk.ca/python/howto/unicode.
PEP100:PythonUnicodeIntegration
4.8codecsCodecregistryandbaseclasses
http://docs.python.org/lib/modulecodecs.html
4.8.2EncodingsandUnicode
Page189
APythonBook
http://docs.python.org/lib/encodingsoverview.html
4.8.3StandardEncodingshttp://docs.python.org/lib/standardencodings.html
ConvertingUnicodeStringsto8bitStrings
http://effbot.org/zone/unicodeconvert.htm
3.4.4Dictionaries
Adictionaryisanunorderedcollectionofkeyvaluepairs.
Adictionaryhasalength,specificallythenumberofkeyvaluepairs.
Adictionaryprovidesfastlookupbykey.
Thekeysmustbeimmutableobjecttypes.
3.4.4.1Literalrepresentationofdictionaries
Curleybracketsareusedtorepresentadictionary.Eachpairinthedictionaryis
representedbyakeyandvalueseparatedbyacolon.Multiplepairsareseparatedby
comas.Forexample,hereisanemptydictionaryandseveraldictionariescontaining
key/valuepairs:
In[4]:d1={}
In[5]:d2={'width':8.5,'height':11}
In[6]:d3={1:'RED',2:'GREEN',3:'BLUE',}
In[7]:d1
Out[7]:{}
In[8]:d2
Out[8]:{'height':11,'width':8.5}
In[9]:d3
Out[9]:{1:'RED',2:'GREEN',3:'BLUE'}
Notes:
Acommaafterthelastpairisoptional.SeetheREDGREENBLUEexample
above.
Stringsandintegersworkaskeys,sincetheyareimmutable.Youmightalsowant
tothinkabouttheuseoftuplesofintegersaskeysinadictionaryusedto
representasparsearray.
Exercises:
1. Defineadictionarythathasthefollowingkeyvaluepairs:
2. Defineadictionarytorepresentthe"enum"daysoftheweek:Sunday,Monday,
Tuesday,...
Solutions:
1. Adictionarywhosekeysandvaluesarestringscanbeusedtorepresentthistable:
Page190
APythonBook
vegetables={
'Eggplant':'Purple',
'Tomato':'Red',
'Parsley':'Green',
'Lemon':'Yellow',
'Pepper':'Green',
}
Notethattheopencurlybracketenablesustocontinuethisstatementacross
multiplelineswithoutusingabackslash.
2. Wemightusestringsforthenamesofthedaysoftheweekaskeys:
DAYS={
'Sunday':1,
'Monday':2,
'Tuesday':3,
'Wednesday':4,
'Thrusday':5,
'Friday':6,
'Saturday':7,
}
3.4.4.2Operatorsondictionaries
Dictionariessupportthefollowing"operators":
Lengthlen(d)returnsthenumberofpairsinadictionary.
IndexingYoucanbothsetandgetthevalueassociatedwithakeybyusingthe
indexingoperator[].Examples:
In[12]:d3[2]
Out[12]:'GREEN'
In[13]:d3[0]='WHITE'
In[14]:d3[0]
Out[14]:'WHITE'
TestforkeyTheinoperatortestsfortheexistenceofakeyinadictionary.
Example:
In[6]:trees={'poplar':'deciduous','cedar':
'evergreen'}
In[7]:if'cedar'intrees:
...:print'Thecedaris%s'%
(trees['cedar'],)
...:
Thecedarisevergreen
Exercises:
1. Createanemptydictionary,thenusetheindexingoperator[]toinsertthe
followingnamevaluepairs:
"red""255:0:0"
Page191
APythonBook
"green""0:255:0"
"blue""0:0:255"
2. Printoutthenumberofitemsinyourdictionary.
Solutions:
1. Wecanuse"[]"tosetthevalueofakeyinadictionary:
deftest():
colors={}
colors["red"]="255:0:0"
colors["green"]="0:255:0"
colors["blue"]="0:0:255"
print'Thevalueofredis"%s"'%
(colors['red'],)
print'Thecolorsdictionarycontains%ditems.'%
(len(colors),)
test()
Whenwerunthis,wesee:
Thevalueofredis"255:0:0"
Thecolorsdictionarycontains3items.
2. Thelen()builtinfunctiongivesusthenumberofitemsinadictionary.Seethe
previoussolutionforanexampleofthis.
3.4.4.3Methodsondictionaries
Hereisatablethatdescribesthemethodsapplicabletodictionarys:
Operation
Result
len(a)
thenumberofitemsina
a[k]
theitemofawithkeyk
a[k]=v
seta[k]tov
dela[k]
removea[k]froma
a.clear()
removeallitemsfroma
a.copy()
a(shallow)copyofa
kina
Trueifahasakeyk,elseFalse
knotina
equivalenttonotkina
a.has_key(k)
equivalenttokina,usethatforminnewcode
a.items()
acopyofa'slistof(key,value)pair
Page192
APythonBook
Operation
Result
a.keys()
acopyofa'slistofkeys
a.update([b])
updatesawithkey/valuepairsfromb,overwritingexisting
keys,returnsNone
a.fromkeys(seq[,value])
createsanewdictionarywithkeysfromseqandvaluessetto
value
a.values()
acopyofa'slistofvalues
a.get(k[,x])
a[k]ifkina,elsex)
a.setdefault(k[,x])
a[k]ifkina,elsex(alsosettingit)
a.pop(k[,x])
a[k]ifkina,elsex(andremovek)(8)
a.popitem()
removeandreturnanarbitrary(key,value)pair
a.iteritems()
returnaniteratorover(key,value)pairs
a.iterkeys()
returnaniteratoroverthemapping'skeys
a.itervalues()
returnaniteratoroverthemapping'svalues
YoucanalsofindthistableatthestandarddocumentationWebsiteinthe"Python
LibraryReference":MappingTypesdicthttp://docs.python.org/lib/typesmapping.html
Exercises:
1. Printthekeysandvaluesintheabove"vegetable"dictionary.
2. Printthekeysandvaluesintheabove"vegetable"dictionarywiththekeysin
alphabeticalorder.
3. Testfortheoccuranceofakeyinadictionary.
Solutions:
1. Wecanusethed.items()methodtoretrievealistoftuplescontaining
keyvaluepairs,thenuseunpackingtocapturethekeyandvalue:
Vegetables={
'Tomato':'Red',
'Parsley':'Green',
'Lemon':'Yellow',
'Pepper':'Green',
}
Page193
APythonBook
deftest():
forkey,valueinVegetables.items():
print'key:',key,'value:',value
test()
2. Weretrievealistofkeyswiththekeys()method,thesortitwiththelist
sort()method:
Vegetables={
'Tomato':'Red',
'Parsley':'Green',
'Lemon':'Yellow',
'Pepper':'Green',
}
deftest():
keys=Vegetables.keys()
keys.sort()
forkeyinkeys:
print'key:',key,'value:',Vegetables[key]
test()
3. Totestfortheexistenceofakeyinadictionary,wecanuseeitherthein
operator(preferred)orthed.has_key()method(oldstyle):
Vegetables={
'Tomato':'Red',
'Parsley':'Green',
'Lemon':'Yellow',
'Pepper':'Green',
}
deftest():
if'Eggplant'inVegetables:
print'wehave%segplants'%
Vegetables['Eggplant']
if'Banana'notinVegetables:
print'yeswehavenobananas'
ifVegetables.has_key('Parsley'):
print'wehaveleafy,%sparsley'%
Vegetables['Parsley']
test()
Whichwillprintout:
wehavePurpleegplants
yeswehavenobananas
wehaveleafy,Greenparsley
Page194
APythonBook
3.4.5Files
APythonfileobjectrepresentsafileonafilesystem.
Afileobjectopenforreadingatextfileisiterable.Whenweiterateoverit,itproduces
thelinesinthefile.
Afilemaybeopenedinthesemodes:
'r'readmode.Thefilemustexist.
'w'writemode.Thefileiscreated;anexistingfileisoverwritten.
'a'appendmode.Anexistingfileisopenedforwriting(attheendofthefile).A
fileiscreatedifitdoesnotexist.
Theopen()builtinfunctionisusedtocreateafileobject.Forexample,thefollowing
code(1)opensafileforwriting,then(2)forreading,then(3)forappending,andfinally
(4)forreadingagain:
deftest(infilename):
#1.Openthefileinwritemode,whichcreatesthefile.
outfile=open(infilename,'w')
outfile.write('line1\n')
outfile.close()
#2.Openthefileforreading.
infile=open(infilename,'r')
forlineininfile:
print'Line:',line.rstrip()
infile.close()
#3.Openthefileinappendmode,andaddalinetotheendof
#thefile.
outfile=open(infilename,'a')
outfile.close()
print''*40
#4.Openthefileinreadmodeoncemore.
forlineininfile:
print'Line:',line.rstrip()
infile.close()
test('tmp.txt')
Exercises:
1. Openatextfileforreading,thenreadtheentirefileasasinglestring,andthen
splitthecontentonnewlinecharacters.
2. Openatextfileforreading,thenreadtheentirefileasalistofstrings,whereeach
stringisonelineinthefile.
3. Openatextfileforreading,theniterateofeachlineinthefileandprintitout.
Page195
APythonBook
Solutions:
1. Usetheopen()builtinfunctiontoopenthefileandcreateafileobject.Usethe
read()methodonthefileobjecttoreadtheentirefile.Usethesplit()or
splitlines()methodstosplitthefileintolines:
>>>infile=open('tmp.txt','r')
>>>content=infile.read()
>>>infile.close()
>>>lines=content.splitlines()
>>>printlines
['line1','line2','line3','']
2. Thef.readlines()methodreturnsalistoflinesinafile:
>>>infile=open('tmp.txt','r')
>>>lines=infile.readlines()
>>>infile.close()
>>>printlines
['line1\n','line2\n','line3\n']
3. Sinceafileobject(openforreading)isitselfaniterator,wecaniterateoveritina
forstatement:
"""
Testiterationoveratextfile.
Usage:
pythontest.pyin_file_name
"""
importsys
forlineininfile:
#Stripoffthenewlinecharacterandany
whitespaceon
#theright.
line=line.rstrip()
#Printonlynonblanklines.
ifline:
printline
infile.close()
defmain():
args=sys.argv[1:]
iflen(args)!=1:
print__doc__
sys.exit(1)
infilename=args[0]
test(infilename)
main()
Page196
APythonBook
Notes:
Thelasttwolinesofthissolutioncheckthe__name__attributeofthe
moduleitselfsothatthemodulewillrunasascriptbutwillnotrunwhenthe
moduleisimportedbyanothermodule.
The__doc__attributeofthemodulegivesusthemodule'sdocstring,which
isthestringdefinedatthetopofthemodule.
sys.argvgivesusthecommandline.And,sys.argv[1:]chopsoffthe
programname,leavinguswiththecommanlinearguments.
3.4.6Afewmiscellaneousdatatypes
3.4.6.1None
Noneisasingleton.ThereisonlyoneinstanceofNone.Usethisvaluetoindicatethe
absenceofanyother"real"value.
TestforNonewiththeidentityoperatoris.
Exercises:
1. Createalist,someofwhoseelementsareNone.Thenwriteaforloopthat
countsthenumberofoccurancesofNoneinthelist.
Solutions:
1. TheidentityoperatorsisandisnotcanbeusedtotestforNone:
>>>a=[11,None,'abc',None,{}]
>>>a
[11,None,'abc',None,{}]
>>>count=0
>>>foritemina:
...ifitemisNone:
...count+=1
...
>>>
>>>printcount
2
3.4.6.2ThebooleansTrueandFalse
PythonhasthetwobooleanvaluesTrueandFalse.Manycomparisonoperatorsreturn
TrueandFalse.
Examples:
1. Whatvalueisreturnedby3>2?
Answer:ThebooleanvalueTrue.
2. Giventhesevariabledefinitions:
Page197
APythonBook
x=3
y=4
z=5
Whatdoesthefollowingprintout:
printy>xandz>y
AnswerPrintsout"True"
3.5Statements
3.5.1Assignmentstatement
Theassignmentstatementusestheassignmentoperator=.
Theassignmentstatementisabindingstatement:itbindsavaluetoanamewithina
namespace.
Exercises:
1. Bindthevalue"eggplant"tothevariablevegetable.
Solutions:
1.The=operatorisanassignmentstatementthatbindsavaluetoavariable:
>>>vegetable="eggplant"
Thereisalsoaugmentedassignmentusingtheoperators+=,=,*=,/=,etc.
Exercises:
1. Useaugmentedassignmenttoincrementthevalueofaninteger.
2. Useaugmentedassignmenttoappendcharacterstotheendofastring.
3. Useaugmentedassignmenttoappendtheitemsinonelisttoanother.
4. Useaugmentedassignmenttodecrementavariablecontaininganintegerby1.
Solutions:
1. The+=operatorincrementsthevalueofaninteger:
>>>count=0
>>>count+=1
>>>count
1
>>>count+=1
>>>count
2
2. The+=operatorappendscharacterstotheendofastring:
>>>buffer='abcde'
>>>buffer+='fgh'
Page198
APythonBook
>>>buffer
'abcdefgh'
3. The+=operatorappendsitemsinonelisttoanother:
In[20]:a=[11,22,33]
In[21]:b=[44,55]
In[22]:a+=b
In[23]:a
Out[23]:[11,22,33,44,55]
1. The=operatordecrementsthevalueofaninteger:
>>>count=5
>>>count
5
>>>count=1
>>>count
4
Youcanalsoassignavalueto(1)anelementofalist,(2)aniteminadictionary,(3)an
attributeofanobject,etc.
Exercises:
1. Createalistofthreeitems,thenassignanewvaluetothe2ndelementinthelist.
2. Createadictionary,thenassignvaluestothekeys"vegetable"and"fruit"inthat
dictionary.
3. Usethefollowingcodetocreateaninstanceofaclass:
classA(object):
pass
a=A()
Thenassignvaluestoanattribuenamedcategoryinthatinstance.
Solutions:
1. Assignmentwiththeindexingoperator[]assignsavaluetoanelementinalist:
>>>trees=['pine','oak','elm']
>>>trees
['pine','oak','elm']
>>>trees[1]='cedar'
>>>trees
['pine','cedar','elm']
2. Assignmentwiththeindexingoperator[]assignsavaluetoanitem(akeyvalue
pair)inadictionary:
>>>foods={}
>>>foods
{}
>>>foods['vegetable']='greenbeans'
>>>foods['fruit']='nectarine'
>>>foods
Page199
APythonBook
{'vegetable':'greenbeans','fruit':'nectarine'}
3. Assignmentalongwiththedereferencingoperator.(dot)enablesustoassigna
valuetoanattributeofanobject:
>>>classA(object):
...pass
...
>>>a=A()
>>>a.category=25
>>>a.__dict__
{'category':25}
>>>a.category
25
3.5.2printstatement
Warning:BeawarethattheprintstatementwillgoawayinPythonversion3.0.Itwill
bereplacedbythebuiltinprint()function.
Theprintstatementsendsoutputtostandardoutput.Itprovidesasomewhatmore
convenientwayofproducingoutputthanusingsys.stdout.write().
Theprintstatementtakesaseriesofzeroormoreobjectsseparatedbycommas.Zero
objectsproducesablankline.
Theprintstatementnormallyaddsanewlineattheendofitsoutput.Toeliminatethat,
addacommaattheend.
Exercises:
1. Printasinglestring.
2. Printthreestringsusingasingleprintstatement.
3. Givenavariablenamecontainingastring,printoutthestringMynameis
"xxxx".,wherexxxxisreplacebythevalueofname.Usethestringformatting
operator.
Solutions:
1. Wecanprintaliteralstring:
>>>print'Hello,there'
Hello,there
2. Wecanprintliteralsandthevalueofvariables:
>>>description='cute'
>>>print'Iama',description,'kid.'
Iamacutekid.
3. Thestringformattingoperatorgivesmorecontroloverformattingoutput:
>>>name='Alice'
Page200
APythonBook
>>>print'Mynameis"%s".'%(name,)
Mynameis"Alice".
3.5.3if:statementexercises
Theifstatementisacompoundstatementthatenablesustoconditionallyexecute
blocksofcode.
Theifstatementalsohasoptionalelif:andelse:clauses.
Theconditioninanif:orelif:clausecanbeanyPythonexpression,inotherwords,
somethingthatreturnsavalue(evenifthatvalueisNone).
Intheconditioninanif:orelif:clause,thefollowingvaluescountas"false":
False
None
Numericzero
Anemptycollection,forexampleanemptylistordictionary
Anemptystring(astringoflengthzero)
Allothervaluescountastrue.
Exercises:
1. Giventhefollowinglist:
>>>bananas=['banana1','banana2','banana3',]
Printonemessageifitisanemptylistandanothermessgeifitisnot.
2. HereisonewayofdefiningaPythonequivalentofan"enum":
NO_COLOR,RED,GREEN,BLUE=range(4)
Writeanif:statementwhichimplementstheeffectofa"switch"statementin
Python.Printoutauniquemessageforeachcolor.
Solutions:
1. Wecantestforanemptyornonemptylist:
>>>bananas=['banana1','banana2','banana3',]
>>>ifnotbananas:
...print'yes,wehavenobananas'
...else:
...print'yes,wehavebananas'
...
yes,wehavebananas
2. Wecansimulatea"switch"statementusingif:elif:...:
NO_COLOR,RED,GREEN,BLUE=range(4)
deftest(color):
Page201
APythonBook
ifcolor==RED:
print"It'sred."
elifcolor==GREEN:
print"It'sgreen."
elifcolor==BLUE:
print"It'sblue."
defmain():
color=BLUE
test(color)
main()
Which,whenrunprintsoutthefollowing:
It'sblue.
3.5.4for:statementexercises
Thefor:statementisthePythonwaytoiterateoverandprocesstheelementsofa
collectionorotheriterable.
Thebasicformofthefor:statementisthefollowing:
forXinY:
statement
o
o
o
where:
Xissomethingthatcanbeassignedto.ItissomethingtowhichPythoncanbinda
value.
Yissomecollectionorotheriterable.
Exercises:
1. Createalistofintegers.Useafor:statementtoprintouteachintegerinthelist.
2. Createastring.printouteachcharacterinthestring.
Solutions:
1. Thefor:statementcaniterateovertheitemsinalist:
In[13]:a=[11,22,33,]
In[14]:forvalueina:
....:print'value:%d'%value
....:
....:
value:11
value:22
value:33
Page202
APythonBook
2. Thefor:statementcaniterateoverthecharactersinastring:
In[16]:b='chocolate'
In[17]:forchr1inb:
....:print'character:%s'%chr1
....:
....:
character:c
character:h
character:o
character:c
character:o
character:l
character:a
character:t
character:e
Notes:
Inthesolution,Iusedthevariablenamechr1ratherthanchrsoasnotto
overwritethenameofthebuiltinfunctionchr().
Whenweneedasequentialindex,wecanusetherange()builtinfunctiontocreatea
listofintegers.And,thexrange()builtinfunctionproducesaninteratorthatproduces
asequenceofintegerswithoutcreatingtheentirelist.Toiterateoveralargesequenceof
integers,usexrange()insteadofrange().
Exercises:
1. Printouttheintegersfrom0to5insequence.
2. Computethesumofalltheintegersfrom0to99999.
3. Giventhefollowinggeneratorfunction:
importurllib
Urls=[
'http://yahoo.com',
'http://python.org',
'http://gimp.org',#TheGNUimagemanipulation
program
]
defwalk(url_list):
forurlinurl_list:
f=urllib.urlopen(url)
stuff=f.read()
f.close()
yieldstuff
Writeafor:statementthatusesthisiteratorgeneratortoprintthelengthsofthe
contentateachoftheWebpagesinthatlist.
Solutions:
1. Therange()builtinfunctiongivesusasequencetoiterateover:
Page203
APythonBook
In[5]:foridxinrange(6):
...:print'idx:%d'%idx
...:
...:
idx:0
idx:1
idx:2
idx:3
idx:4
idx:5
2. Sincethatsequenceisabitlarge,we'llusexrange()insteadofrange():
In[8]:count=0
In[9]:forninxrange(100000):
...:count+=n
...:
...:
In[10]:count
Out[10]:4999950000
3. Thefor:statementenablesustoiterateoveriterablesaswellascollections:
importurllib
Urls=[
'http://yahoo.com',
program
]
defwalk(url_list):
forurlinurl_list:
stuff=f.read()
f.close()
yieldstuff
deftest():
forurlinwalk(Urls):
print'length:%d'%(len(url),)
test()
WhenIranthisscript,itprintsthefollowing:
length:9562
length:16341
length:12343
Ifyouneedanindexwhileiteratingoverasequence,considerusingtheenumerate()
builtinfunction.
Page204
APythonBook
Exercises:
1. Giventhefollowingtwolistsofintegersofthesamelength:
a=[1,2,3,4,5]
b=[100,200,300,400,500]
Addthevaluesinthefirstlisttothecorrespondingvaluesinthesecondlist.
Solutions:
1. Theenumerate()builtinfunctiongivesusanindexandvaluesfroma
sequence.Sinceenumerate()givesusaninteratorthatproducesasequenceof
twotuples,wecanunpackthosetuplesintoindexandvaluevariablesinthe
headerlineoftheforstatement:
In[13]:a=[1,2,3,4,5]
In[14]:b=[100,200,300,400,500]
In[15]:
In[16]:foridx,valueinenumerate(a):
....:b[idx]+=value
....:
....:
In[17]:b
Out[17]:[101,202,303,404,505]
3.5.5while:statementexercises
Awhile:statementexecutesablockofcoderepeatedlyaslongasaconditionistrue.
Hereisatemplateforthewhile:statement:
whilecondition:
statement
o
o
o
Where:
conditionisanexpression.Theexpressionissomethingthatreturnsavalue
whichcanbeinterpretedastrueorfalse.
Exercises:
1. Writeawhile:loopthatdoublesallthevaluesinalistofintegers.
Solutions:
1. Awhile:loopwithanindexvariablecanbeusedtomodifyeachelementofa
list:
deftest_while():
numbers=[11,22,33,44,]
print'before:%s'%(numbers,)
Page205
APythonBook
idx=0
whileidx<len(numbers):
numbers[idx]*=2
idx+=1
print'after:%s'%(numbers,)
But,noticethatthistaskiseasierusingthefor:statementandthebuiltin
enumerate()function:
deftest_for():
numbers=[11,22,33,44,]
foridx,iteminenumerate(numbers):
numbers[idx]*=2
3.5.6breakandcontinuestatements
Thecontinuestatementskipstheremainderofthestatementsinthebodyofaloop
andstartsimmediatelyatthetopoftheloopagain.
Abreakstatementinthebodyofaloopterminatestheloop.Itexitsfromthe
immediatelycontainingloop.
breakandcontinuecanbeusedinbothfor:andwhile:statements.
Exercises:
1. Writeafor:loopthattakesalistofintegersandtripleseachintegerthatiseven.
Usethecontinuestatement.
2. Writealoopthattakesalistofintegersandcomputesthesumofalltheintegers
upuntilazeroisfoundinthelist.Usethebreakstatement.
Solutions:
1. Thecontinuestatementenablesusto"skip"itemsthatsatisfyaconditionor
test:
deftest():
numbers=[11,22,33,44,55,66,]
foridx,iteminenumerate(numbers):
ifitem%2!=0:
continue
numbers[idx]*=3
test()
2. Thebreakstatementenablesustoexitfromaloopwhenwefindazero:
deftest():
numbers=[11,22,33,0,44,55,66,]
Page206
APythonBook
print'numbers:%s'%(numbers,)
sum=0
foriteminnumbers:
ifitem==0:
break
sum+=item
print'sum:%d'%(sum,)
test()
3.5.7Exceptionsandthetry:except:andraisestatements
Thetry:except:statementenablesustocatchanexceptionthatisthrownfrom
withinablockofcode,orfromcodecalledfromanydepthwithingthatblock.
Theraisestatementenablesustothrowanexception.
Anexceptionisaclassoraninstanceofanexceptionclass.Ifanexceptionisnotcaught,
itresultsinatracebackandterminationoftheprogram.
Thereisasetofstandardexceptions.Youcanlearnaboutthemhere:BuiltinExceptions
http://docs.python.org/lib/moduleexceptions.html.
Youcandefineyourownexceptionclasses.Todoso,createanemptysubclassofthe
classException.Definingyourownexceptionwillenableyou(orothers)tothrow
andthencatchthatspecificexceptiontypewhileignoreothersexceptions.
Exercises:
1. Writeatry:except:statementthatattemptstoopenafileforreadingand
catchestheexceptionthrownwhenthefiledoesnotexist.
Question:Howdoyoufindoutthenameoftheexceptionthatisthrownforan
input/outputerrorsuchasthefailuretoopenafile?
2. Defineanexceptionclass.Thenwriteatry:except:statementinwhichyou
throwandcatchthatspecificexception.
3. Defineanexceptionclassanduseittoimplementamultilevelbreakfroman
innerloop,bypassinganouterloop.
Solutions:
1. UsethePythoninteractiveinterpretertolearntheexceptiontypethrownwhena
I/Oerroroccurs.Example:
>>>infile=open('xx_nothing__yy.txt','r')
File"<stdin>",line1,in<module>
IOError:[Errno2]Nosuchfileordirectory:
'xx_nothing__yy.txt'
>>>
Page207
APythonBook
Inthiscase,theexceptiontypeisIOError.
Now,writeatry:except:blockwhichcatchesthatexception:
deftest():
infilename='nothing_noplace.txt'
try:
forlineininfile:
printline
exceptIOError,exp:
print'cannotopenfile"%s"'%infilename
test()
2. WedefineaexceptionclassasasubclassofclassException,thenthrowit
(withtheraisestatement)andcatchit(withatry:except:statement):
classSizeError(Exception):
pass
deftest_exception(size):
try:
ifsize<=0:
raiseSizeError,'sizemustbegreaterthan
zero'
#Produceadifferenterrortoshowthatit
willnotbecaught.
x=y
exceptSizeError,exp:
print'%s'%(exp,)
print'goodbye'
deftest():
test_exception(1)
print''*40
test_exception(1)
test()
Whenwerunthisscript,itproducesthefollowingoutput:
sizemustbegreaterthanzero
goodbye
File"workbook027.py",line20,in<module>
test()
File"workbook027.py",line18,intest
test_exception(1)
File"workbook027.py",line10,intest_exception
x=y
NameError:globalname'y'isnotdefined
Page208
APythonBook
Notes:
Ourexcept:clausecaughttheSizeError,butallowedtheNameError
tobeuncaught.
3. WedefineasubclassofofclassException,thenraiseitinaninnerloopand
catchitoutsideofanouterloop:
classBreakException1(Exception):
pass
deftest():
a=[11,22,33,44,55,66,]
b=[111,222,333,444,555,666,]
try:
forxina:
print'outerx:%d'%x
foryinb:
ifx>22andy>444:
raiseBreakException1('leaving
innerloop')
print'innery:%d'%y
print'outerafter'
print''*40
exceptBreakException1,exp:
print'outofloopexp:%s'%exp
test()
Hereiswhatthisprintsoutwhenrun:
outerx:11
innery:111
innery:222
innery:333
innery:444
innery:555
innery:666
outerafter
outerx:22
innery:111
innery:222
innery:333
innery:444
innery:555
innery:666
outerafter
outerx:33
innery:111
innery:222
innery:333
innery:444
outofloopexp:leavinginnerloop
Page209
APythonBook
3.6Functions
Afunctionhasthesecharacteristics:
Itgroupsablockofcodetogethersothatwecancallitbyname.
Itenablesustopassvaluesintothethefunctionwhenwecallit.
Itcanreturnsavalue(evenifNone).
Whenafunctioniscalled,ithasitsownnamespace.Variablesinthefunctionare
localtothefunction(anddisappearwhenthefunctionexits).
Afunctionisdefinedwiththedef:statement.Hereisasimpleexample/template:
deffunction_name(arg1,arg2):
local_var1=arg1+1
local_var2=arg2*2
returnlocal_var1+local_var2
And,hereisanexampleofcallingthisfunction:
result=function_name(1,2)
Hereareafewnotesofexplanation:
Theabovedefinesafunctionwhosenameisfunction_name.
Thefunctionfunction_namehastwoarguments.Thatmeansthatwecanand
mustpassinexactlytwovalueswhenwecallit.
Thisfunctionhastwolocalvariables,local_var1andlocal_var2.These
variablesarelocalinthesensethatafterwecallthisfunction,thesetwovariables
arenotavailableinthelocationofthecaller.
Whenwecallthisfunction,itreturnsonevalue,specificallythesumof
local_var1andlocal_var2.
Exercises:
1. Writeafunctionthattakesalistofintegersasanargument,andreturnsthesum
oftheintegersinthatlist.
Solutions:
1. Thereturnstatementenablesustoreturnavaluefromafunction:
deflist_sum(values):
sum=0
forvalueinvalues:
sum+=value
returnsum
deftest():
a=[11,22,33,44,]
printlist_sum(a)
Page210
APythonBook
test()
3.6.1Optionalargumentsanddefaultvalues
Youcanprovideadefaultvalueforanargumenttoafunction.
Ifyoudo,thatargumentisoptional(whenthefunctioniscalled).
Hereareafewthingstolearnaboutoptionalarguments:
Provideadefaultvaluewithanequalsignandavalue.Example:
defsample_func(arg1,arg2,arg3='empty',arg4=0):
Allparameterswithdefaultvaluesmustbeafter(totherightof)normal
parameters.
Donotuseamutableobjectasadefaultvalue.Becausethedef:statementis
evaluatedonlyonceandnoteachtimethefunctioniscalled,themutableobject
mightbesharedacrossmultiplecallstothefunction.Donotdothis:
defsample_func(arg1,arg2=[]):
Instead,dothis:
defsample_func(arg1,arg2=None):
ifarg2isNone:
arg2=[]
Hereisanexamplethatillustrateshowthismightgowrong:
defadder(a,b=[]):
b.append(a)
returnb
deftest():
printadder('aaa')
printadder('bbb')
printadder('ccc')
test()
Which,whenexecuted,displaysthefollowing:
['aaa']
['aaa','bbb']
['aaa','bbb','ccc']
Exercises:
1. Writeafunctionthatwritesastringtoafile.Thefunctiontakestwoarguments:
(1)afilethatisopenforoutputand(2)astring.Givethesecondargument(the
string)adefaultvaluesothatwhenthesecondargumentisomitted,anempty,
blanklineiswrittentothefile.
Page211
APythonBook
2. Writeafunctionthattakesthefollowingarguments:(1)aname,(2)avalue,and
(3)andoptionaldictionary.Thefunctionaddsthevaluetothedictionaryusingthe
nameasakeyinthedictionary.
Solutions:
1. Wecanpassafileaswewouldanyotherobject.And,wecanuseanewline
characterasadefaultparametervalue:
importsys
defwriter(outfile,msg='\n'):
outfile.write(msg)
deftest():
writer(sys.stdout,'aaaaa\n')
writer(sys.stdout)
writer(sys.stdout,'bbbbb\n')
test()
Whenrunfromthecommandline,thisprintsoutthefollowing:
aaaaa
bbbbb
2. Inthissolutionwearecarefulnottouseamutableobjectasadefaultvalue:
defadd_to_dict(name,value,dic=None):
ifdicisNone:
dic={}
dic[name]=value
returndic
deftest():
dic1={'albert':'cute',}
printadd_to_dict('barry','funny',dic1)
printadd_to_dict('charlene','smart',dic1)
printadd_to_dict('darryl','outrageous')
printadd_to_dict('eddie','friendly')
test()
Ifwerunthisscript,wesee:
{'barry':'funny','albert':'cute'}
{'barry':'funny','albert':'cute','charlene':
'smart'}
{'darryl':'outrageous'}
{'eddie':'friendly'}
Notes:
It'simportantthatthedefaultvalueforthedictionaryisNoneratherthanan
emptydictionary,forexample({}).Rememberthatthedef:statementis
Page212
APythonBook
evaluatedonlyonce,whichresultsinasingledictionary,whichwouldbe
sharedbyallcallersthatdonotprovideadictionaryasanargument.
3.6.2Passingfunctionsasarguments
Afunction,likeanyotherobject,canbepassedasanargumenttoafunction.Thisisdue
thethefactthatalmostall(maybeall)objectsinPythonare"firstclassobjects".Afirst
classobjectisonewhichwecan:
1. Storeinadatastructure(e.g.alist,adictionary,...).
2. Passtoafunction.
3. Returnfromafunction.
Exercises:
1. Writeafunctionthattakesthreearguments:(1)aninputfile,(2)anoutputfile,
and(3)afilterfunction:
Argument1isafileopenedforreading.
Argument2isafileopenedforwriting.
Argument3isafunctionthattakesasingleargument(astring),performsa
transformationonthatstring,andreturnsthetransformedstring.
Theabovefunctionshouldreadeachlineintheinputtextfile,passthatline
throughthefilterfunction,thenwritethat(possibly)transformedlinetothe
outputfile.
Now,writeoneormore"filterfunctions"thatcanbepassedtothefunction
describedabove.
Solutions:
1. Thisscriptaddsorremovescommentcharacterstothelinesofafile:
importsys
deffilter(infile,outfile,filterfunc):
forlineininfile:
line=filterfunc(line)
outfile.write(line)
defadd_comment(line):
line='##%s'%(line,)
returnline
defremove_comment(line):
ifline.startswith('##'):
line=line[3:]
returnline
defmain():
filter(sys.stdin,sys.stdout,add_comment)
Page213
APythonBook
main()
Runningthismightproducesomethinglikethefollowing(noteforMSWindows
users:usetypeinsteadofcat):
$cattmp.txt
line1
line2
line3
$cattmp.txt|pythonworkbook005.py
##line1
##line2
##line3
3.6.3Extraargsandkeywordargs
Additionalpositionalargumentspassedtoafunctionthatarenotspecifiedinthefunction
definition(thedef:statement``),arecollectedinanargumentprecededbyasingle
asterisk.Keywordargumentspassedtoafunctionthatarenotspecifiedinthefunction
definitioncanbecollectedinadictionaryandpassedtoanargumentprecededbya
doubleasterisk.
Examples:
1. Writeafunctionthattakesonepositionalargument,oneargumentwithadefault
value,andalsoextraargsandkeywordargs.
2. Writeafunctionthatpassesallitsarguments,nomatterhowmany,toacallto
anotherfunction.
Solutions:
1. Weuse*argsand**kwargstocollectextraargumentsandextrakeyword
arguments:
defshow_args(x,y=1,*args,**kwargs):
print''*40
print'x:',x
print'y:',y
print'args:',args
print'kwargs:',kwargs
deftest():
show_args(1)
show_args(x=2,y=3)
show_args(y=5,x=4)
show_args(4,5,6,7,8)
show_args(11,y=44,a=55,b=66)
test()
Page214
APythonBook
Runningthisscriptproducesthefollowing:
x:1
y:1
args:()
kwargs:{}
x:2
y:3
args:()
kwargs:{}
x:4
y:5
args:()
kwargs:{}
x:4
y:5
args:(6,7,8)
kwargs:{}
x:11
y:44
args:()
kwargs:{'a':55,'b':66}
Notes:
Thespellingofargsandkwargsisnotfixed,butthe
2. Weuseargsandkwargstocatchandpassonallarguments:
deffunc1(*args,**kwargs):
print'args:%s'%(args,)
print'kwargs:%s'%(kwargs,)
deffunc2(*args,**kwargs):
print'before'
func1(*args,**kwargs)
print'after'
deftest():
func2('aaa','bbb','ccc',arg1='ddd',arg2='eee')
test()
Whenwerunthis,itprintsthefollowing:
before
args:('aaa','bbb','ccc')
kwargs:{'arg1':'ddd','arg2':'eee'}
after
Page215
APythonBook
Notes:
Inafunctioncall,the*operatorunrollsalistintoindividualpositional
arguments,andthe**operatorunrollsadictionaryintoindividualkeyword
arguments.
3.6.3.1Orderofarguments(positional,extra,andkeywordargs)
Inafunctiondefinition,argumentsmustappearinthefollowingorder,fromlefttoright:
1. Positional(normal,plain)arguments
2. Argumentswithdefaultvalues,ifany
3. Extraargumentsparameter(procededbysingleasterisk),ifpresent
4. Keywordargumentsparameter(procededbydoubleasterisk),ifpresent
Inafunctioncall,argumentsmustappearinthefollowingorder,fromlefttoright:
1. Positional(plain)arguments
2. Extraarguments,ifpresent
3. Keywordarguments,ifpresent
3.6.4Functionsandducktypingandpolymorphism
Iftheargumentsandreturnvalueofafunctionsatisfysomedescription,thenwecansay
thatthefunctionispolymorphicwithrespecttothatdescription.
Ifthesomeofthemethodsofanobjectsatisfysomedescription,thenwecansaythatthe
objectispolymorphicwithrespecttothatdescription.
Basically,whatthisdoesistoenableustouseafunctionoranobjectanywherethat
functionsatisfiestherequirementsgivenbyadescription.
Exercises:
1. Implementafunctionthattakestwoarguments:afunctionandanobject.It
appliesthefunctionargumenttotheobject.
2. Implementafunctionthattakestwoarguments:alistoffunctionsandanobject.
Itapplieseachfunctioninthelisttotheargument.
Solutions:
1. Wecanpassafunctionasanargumenttoafunction:
deffancy(obj):
print'fancyfancy%sfancyfancy'%(obj,)
defplain(obj):
print'plain%splain'%(obj,)
defshow(func,obj):
func(obj)
Page216
APythonBook
defmain():
a={'aa':11,'bb':22,}
show(fancy,a)
show(plain,a)
main()
2. Wecanalsoputfunctions(functionobjects)inadatastructure(forexample,a
list),andthenpassthatdatastructuretoafunction:
deffancy(obj):
print'fancyfancy%sfancyfancy'%(obj,)
defplain(obj):
print'plain%splain'%(obj,)
Func_list=[fancy,plain,]
defshow(funcs,obj):
forfuncinfuncs:
func(obj)
defmain():
a={'aa':11,'bb':22,}
show(Func_list,a)
main()
NoticethatPythonsupportspolymorphism(withor)withoutinheritance.Thistypeof
polymorphismisenabledbywhatiscalledducktyping.Formoreonthissee:Duck
typinghttp://en.wikipedia.org/wiki/Duck_typingatWikipedia.
3.6.5Recursivefunctions
Arecursivefunctionisafunctionthatcallsitself.
Arecursivefunctionmusthavealimitingcondition,orelseitwillloopendlessly.
Eachrecursivecallconsumesspaceonthefunctioncallstack.Therefore,thenumberof
recursionsmusthavesomereasonableupperbound.
Exercises:
1. Writearecursivefunctionthatprintsinformationabouteachnodeinthe
followingtreestructuredatastructure:
Tree={
'name':'animals',
'left_branch':{
Page217
APythonBook
'name':'birds',
'left_branch':{
'name':'seedeaters',
'left_branch':{
'name':'housefinch',
'left_branch':None,
'right_branch':None,
},
'right_branch':{
'name':'whitecrownedsparrow',
'left_branch':None,
},
},
'right_branch':{
'name':'insecteaters',
'left_branch':{
'name':'hermitthrush',
'left_branch':None,
},
'right_branch':{
'name':'blackheadedphoebe',
'left_branch':None,
},
},
},
}
Solutions:
1. Wewritearecursivefunctiontowalkthewholetree.Therecursivefunctioncalls
itselftoprocesseachchildofanodeinthetree:
Tree={
'name':'animals',
'left_branch':{
'name':'birds',
'left_branch':{
'name':'seedeaters',
'left_branch':{
'name':'housefinch',
'left_branch':None,
},
'right_branch':{
'name':'whitecrownedsparrow',
'left_branch':None,
},
},
Page218
APythonBook
'right_branch':{
'name':'insecteaters',
'left_branch':{
'name':'hermitthrush',
'left_branch':None,
},
'right_branch':{
'name':'blackheadedphoebe',
'left_branch':None,
},
},
},
}
Indents=[''*idxforidxinrange(10)]
defwalk_and_show(node,level=0):
ifnodeisNone:
return
print'%sname:%s'%(Indents[level],node['name'],
)
level+=1
walk_and_show(node['left_branch'],level)
walk_and_show(node['right_branch'],level)
deftest():
walk_and_show(Tree)
test()
Notes:
Later,youwilllearnhowtocreateequivalentdatastructuresusingclassesand
OOP(objectorientedprogramming).FormoreonthatseeRecursivecallsto
methodsinthisdocument.
3.6.6Generatorsanditerators
The"iteratorprotocol"defineswhataniteratorobjectmustdoinordertobeusableinan
"iteratorcontext"suchasaforstatement.Theiteratorprotocolisdescribedinthe
standardlibraryreference:IteratorTypeshttp://docs.python.org/lib/typeiter.html
Aneasywaytodefineanobjectthatobeystheiteratorprotocolistowriteagenerator
function.Ageneratorfunctionisafunctionthatcontainsoneormoreyieldstatements.
Ifafunctioncontainsatleastoneyieldstatement,thenthatfunctionwhencalled,
returnsgeneratoriterator,whichisanobjectthatobeystheiteratorprotocol,i.e.it'san
iteratorobject.
Page219
APythonBook
NotethatinrecentversionsofPython,yieldisanexpression.Thisenablestheconsumer
tocommunicatebackwiththeproducer(thegeneratoriterator).Formoreonthis,see
PEP:342CoroutinesviaEnhancedGenerators
http://www.python.org/dev/peps/pep0342/.
Exercises:
1. ImplementageneratorfunctionThegeneratorproducedshouldyieldall
valuesfromalist/iterablethatsatisfyapredicate.Itshouldapplythetransforms
beforereturneachvalue.Thefunctiontakesthesearguments:
1. valuesAlistofvalues.Actually,itcouldbeanyiterable.
2. predicateAfunctionthattakesasingleargument,performsateston
thatvalue,andreturnsTrueorFalse.
3. transforms(optional)Alistoffunctions.Applyeachfunctioninthislist
andreturnstheresultingvalue.So,forexample,ifthefunctioniscalledlike
this:
result=transforms([11,22],p,[f,g])
thentheresultinggeneratormightreturn:
g(f(11))
2. ImplementageneratorfunctionthattakesalistofURLsasitsargumentand
generatesthecontentsofeachWebpage,onebyone(thatis,itproducesa
sequenceofstrings,theHTMLpagecontents).
Solutions:
1. Hereistheimplementationofafunctionwhichcontainsyield,and,therefore,
producesagenerator:
"""
filter_and_transform
filter_and_transform(content,test_func,
transforms=None)
Returnageneratorthatreturnsitemsfromcontent
afterapplying
thefunctionsintransformsiftheitemsatisfies
test_func.
Arguments:
1.``values`Àlistofvalues
2.``predicate`Àfunctionthattakesasingle
argument,
performsatestonthatvalue,andreturnsTrue
Page220
APythonBook
orFalse.
3.``transforms``(optional)Alistoffunctions.
Applyeach
functioninthislistandreturnstheresulting
value.So,
forexample,ifthefunctioniscalledlike
this::
result=filter_and_transforms([11,22],p,[f,
g])
thentheresultinggeneratormightreturn::
g(f(11))
"""
deffilter_and_transform(content,test_func,
transforms=None):
forxincontent:
iftest_func(x):
iftransformsisNone:
yieldx
elifisiterable(transforms):
forfuncintransforms:
x=func(x)
yieldx
else:
yieldtransforms(x)
defisiterable(x):
flag=True
try:
x=iter(x)
exceptTypeError,exp:
flag=False
returnflag
defiseven(n):
returnn%2==0
deff(n):
returnn*2
defg(n):
returnn**2
deftest():
data1=[11,22,33,44,55,66,77,]
forvalinfilter_and_transform(data1,iseven,f):
print'val:%d'%(val,)
print''*40
forvalinfilter_and_transform(data1,iseven,[f,
Page221
APythonBook
g]):
print''*40
forvalinfilter_and_transform(data1,iseven):
test()
Notes:
Becausefunctionfilter_and_transformcontainsyield,when
called,itreturnsaniteratorobject,whichwecanuseinaforstatement.
Thesecondparameteroffunctionfilter_and_transformtakesany
functionwhichtakesasingleargumentandreturnsTrueorFalse.Thisisan
exampleofpolymorphismand"ducktyping"(seeDuckTyping
http://en.wikipedia.org/wiki/Duck_typing).Ananalogousclaimcanbemade
aboutthethirdparameter.
2. Thefollowingfunctionusestheurllibmoduleandtheyieldfunctionto
generatethecontentsofasequenceofWebpages:
importurllib
Urls=[
'http://yahoo.com',
program
]
defwalk(url_list):
forurlinurl_list:
stuff=f.read()
f.close()
yieldstuff
deftest():
forxinwalk(Urls):
print'length:%d'%(len(x),)
test()
WhenIrunthis,Isee:
$pythongenerator_example.py
length:9554
length:16748
length:11487
Page222
APythonBook
3.7Objectorientedprogrammingandclasses
ClassesprovidePython'swaytodefinenewdatatypesandtodoOOP(objectoriented
programming).
Ifyouhavemadeitthisfar,youhavealreadyusedlotsofobjects.Youhavebeena
"consumer"ofobjectsandtheirservices.Now,youwilllearnhowtodefineand
implementnewkindsofobjects.Youwillbecomea"producer"ofobjects.Youwill
definenewclassesandyouwillimplementthecapabilities(methods)ofeachnewclass.
Aclassisdefinedwiththeclassstatement.Thefirstlineofaclassstatementisa
header(ithasacolonattheend),anditspecifiesthenameoftheclassbeingdefinedand
an(optional)superclass.Andthatheaderintroducesacompoundstatement:specifically,
thebodyoftheclassstatementwhichcontainsindented,nestedstatements,
importantly,defstatementsthatdefinethemethodsthatcanbecalledoninstancesofthe
objectsimplementedbythisclass.
Exercises:
1. Defineaclasswithonemethodshow.Thatmethodshouldprintout"Hello".
Then,createaninstanceofyourclass,andcalltheshowmethod.
Solutions:
1. Asimpleinstancemethodcanhavetheselfparameterandnoothers:
classDemo(object):
defshow(self):
print'hello'
deftest():
a=Demo()
a.show()
test()
Notes:
Noticethatweuseobjectasasuperclass,becausewewanttodefinean
"newstyle"classandbecausethereisnootherclassthatwewantasa
superclass.Seethefollowingformoreinformationonnewstyleclasses:
NewstyleClasseshttp://www.python.org/doc/newstyle/.
InPython,wecreateaninstanceofaclassbycallingtheclass,thatis,we
applythefunctioncalloperator(parentheses)totheclass.
3.7.1Theconstructor
Aclasscandefinemethodswithspecialnames.Youhaveseemsomeofthesebefore.
Thesenamesbeginandendwithadoubleunderscore.
Page223
APythonBook
Oneimportantspecialnameis__init__.It'stheconstructorforaclass.Itiscalled
eachtimeaninstanceoftheclassiscreated.Implementingthismethodinaclassgivesus
achancetoinitializeeachinstanceofourclass.
Exercises:
1. ImplementaclassnamedPlantthathasaconstructorwhichinitializestwo
instancevariables:nameandsize.Also,inthisclass,implementamethod
namedshowthatprintsoutthevaluesoftheseinstancevariables.Createseveral
instancesofyourclassand"show"them.
2. ImplementaclassnameNodethathastwoinstancevariables:dataand
children,wheredataisany,arbitraryobjectandchildrenisalistofchild
Nodes.Alsoimplementamethodnamedshowthatrecursivelydisplaysthe
nodesina"tree".Createaninstanceofyourclassthatcontainsseveralchild
instancesofyourclass.Calltheshowmethodontheroot(topmost)objectto
showthetree.
Solutions:
1. Theconstructorforaclassisamethodwiththespecialname__init__:
classPlant(object):
def__init__(self,name,size):
self.name=name
self.size=size
defshow(self):
print'name:"%s"size:%d'%(self.name,
self.size,)
deftest():
p1=Plant('Eggplant',25)
p2=Plant('Tomato',36)
plants=[p1,p2,]
forplantinplants:
plant.show()
test()
Notes:
Ourconstructortakestwoarguments:nameandsize.Itsavesthosetwo
valuesasinstancevariables,thatisinattributesoftheinstance.
Theshow()methodprintsoutthevalueofthosetwoinstancevariables.
2. Itisagoodideatoinitializeallinstancevariablesintheconstructor.Thatenables
someonereadingourcodetolearnaboutalltheinstancevariablesofaclassby
lookinginasinglelocation:
#simple_node.py
Indents=[''*nforninrange(10)]
Page224
APythonBook
classNode(object):
def__init__(self,name=None,children=None):
self.name=name
ifchildrenisNone:
self.children=[]
else:
defshow_name(self,indent):
print'%sname:"%s"'%(Indents[indent],
self.name,)
defshow(self,indent=0):
self.show_name(indent)
indent+=1
child.show(indent)
deftest():
n1=Node('N1')
n2=Node('N2')
n3=Node('N3')
n4=Node('N4')
n5=Node('N5',[n1,n2,])
n6=Node('N6',[n3,n4,])
n7=Node('N7',[n5,n6,])
n7.show()
test()
Notes:
Noticethatwedonotusetheconstructorforalist([])asadefaultvaluefor
thechildrenparameteroftheconstructor.Alistismutableandwouldbe
createdonlyonce(whentheclassstatementisexecuted)andwouldbeshared.
3.7.2InheritanceImplementingasubclass
Asubclassextendsorspecializesasuperclassbyaddingadditionalmethodstothe
superclassandbyoverridingmethods(withthesamename)thatalreadyexistinthe
superclass.
Exercises:
1. ExtendyourNodeexerciseabovebyaddingtwoadditionalsubclassesofthe
Nodeclass,onenamedPlantandtheothernamedAnimal.ThePlantclass
alsohasaheightinstancevariableandtheAnimalclassalsohasacolor
instancevariable.
Solutions:
1. WecanimportourpreviousNodescript,thenimplementclassesthathavethe
Nodeclassasasuperclass:
Page225
APythonBook
fromsimple_nodeimportNode,Indents
classPlant(Node):
def__init__(self,name,height=1,children=None):
Node.__init__(self,name,children)
self.height=height
print'%sheight:%s'%(Indents[indent],
self.height,)
indent+=1
child.show(indent)
classAnimal(Node):
def__init__(self,name,color='nocolor',
children=None):
Node.__init__(self,name,children)
self.color=color
print'%scolor:"%s"'%(Indents[indent],
self.color,)
indent+=1
child.show(indent)
deftest():
n1=Animal('scrubjay','grayblue')
n2=Animal('raven','black')
n3=Animal('americankestrel','brown')
n4=Animal('redshoulderedhawk','brownand
gray')
n5=Animal('corvid','none',[n1,n2,])
n6=Animal('raptor',children=[n3,n4,])
n7a=Animal('bird',children=[n5,n6,])
n1=Plant('valleyoak',50)
n2=Plant('canyonliveoak',40)
n3=Plant('jefferypine',120)
n4=Plant('ponderosapine',140)
n5=Plant('oak',children=[n1,n2,])
n6=Plant('conifer',children=[n3,n4,])
n7b=Plant('tree',children=[n5,n6,])
n8=Node('birdsandtrees',[n7a,n7b,])
n8.show()
test()
Notes:
TheshowmethodinclassPlantcallstheshow_namemethodinits
superclassusingself.show_name(...).Pythonsearchesupthe
Page226
APythonBook
inheritancetreetofindtheshow_namemethodinclassNode.
Theconstructor(__init__)inclassesPlantandAnimaleachcallthe
constructorinthesuperclassbyusingthenameofthesuperclass.Whythe
difference?Because,if(inthePlantclass,forexample)itused
self.__init__(...)itwouldbecallingthe__init__inthePlant
class,itself.So,itbypassesitselfbyreferencingtheconstructorinthe
superclassdirectly.
Thisexercisealsodemonstrates"polymorphism"Theshowmethodis
calledanumberoftimes,butwhichimplementationexecutesdependson
whichinstanceitiscalledon.Callingontheshowmethodonaninstanceof
classPlantresultsinacalltoPlant.show.Callingtheshowmethodon
aninstanceofclassAnimalresultsinacalltoAnimal.show.Andsoon.It
isimportantthateachshowmethodtakesthecorrectnumberofarguments.
3.7.3Classesandpolymorphism
Pythonalsosupportsclassbasedpolymorphism,whichwas,bytheway,demonstratedin
thepreviousexample.
Exercises:
1. Writethreeclasses,eachofwhichimplementashow()methodthattakesone
argument,astring.Theshowmethodshouldprintoutthenameoftheclassand
themessage.Thencreatealistofinstancesandcalltheshow()methodoneach
objectinthelist.
Solution:
1. Weimplementthreesimpleclassesandthencreatealistofinstancesofthese
classes:
classA(object):
defshow(self,msg):
print'classAmsg:"%s"'%(msg,)
classB(object):
defshow(self,msg):
print'classBmsg:"%s"'%(msg,)
classC(object):
defshow(self,msg):
print'classCmsg:"%s"'%(msg,)
deftest():
objs=[A(),B(),C(),A(),]
foridx,objinenumerate(objs):
msg='message#%d'%(idx+1,)
obj.show(msg)
Page227
APythonBook
test()
Notes:
Wecancalltheshow()methodinanyobjectinthelistobjsaslongaswe
passinasingleparameter,thatis,aslongasweobeytherequirementsof
ducktyping.Wecandothisbecauseallobjectsinthatlistimplementa
show()method.
Inastaticallytypedlanguage,thatisalanguagewherethetypeis(also)
presentinthevariable,alltheinstancesinexamplewouldhavetodescend
fromacommonsuperclassandthatsuperclasswouldhavetoimplementa
show()method.Pythondoesnotimposethisrestriction.And,because
variablesarenotnottypedinPython,perhapsthatwouldnotevenpossible.
Noticethatthisexampleofpolymorphismworkseventhoughthesethree
classes(A,B,andC)arenotrelated(forexample,inaclasshierarchy).All
thatisrequiredforpolymorphismtoworkinPythonisforthemethodnames
tobethesameandtheargumentstobecompatible.
3.7.4Recursivecallstomethods
Amethodinaclasscanrecusivelycallitself.Thisisverysimilartothewayinwhichwe
implementedrecursivefunctionssee:Recursivefunctions.
Exercises:
1. ReimplementthebinarytreeofanimalsandbirdsdescribedinRecursive
functions,butthistime,useaclasstorepresenteachnodeinthetree.
2. Solvethesameproblem,butthistimeimplementatreeinwhicheachnodecan
haveanynumberofchildren(ratherthanexactly2children).
Solutions:
1. Weimplementaclasswiththreeinstancevariables:(1)name,(2)leftbranch,and
(3)rightbranch.Then,weimplementashow()methodthatdisplaysthename
andcallsitselftoshowthechildrenineachsubtree:
Indents=[''*idxforidxinrange(10)]
classAnimalNode(object):
def__init__(self,name,left_branch=None,
right_branch=None):
self.name=name
self.left_branch=left_branch
self.right_branch=right_branch
defshow(self,level=0):
print'%sname:%s'%(Indents[level],
Page228
APythonBook
self.name,)
level+=1
ifself.left_branchisnotNone:
self.left_branch.show(level)
ifself.right_branchisnotNone:
self.right_branch.show(level)
Tree=AnimalNode('animals',
AnimalNode('birds',
AnimalNode('seedeaters',
AnimalNode('housefinch'),
AnimalNode('whitecrownedsparrow'),
),
AnimalNode('insecteaters',
AnimalNode('hermitthrush'),
AnimalNode('blackheadedphoebe'),
),
),
None,
)
deftest():
Tree.show()
test()
2. Insteadofusingaleftbranchandarightbranch,inthissolutionweusealistto
representthechildrenofanode:
classAnimalNode(object):
def__init__(self,data,children=None):
self.data=data
ifchildrenisNone:
self.children=[]
else:
defshow(self,level=''):
print'%sdata:%s'%(level,self.data,)
level+=''
child.show(level)
Tree=AnimalNode('animals',[
AnimalNode('birds',[
AnimalNode('seedeaters',[
AnimalNode('housefinch'),
AnimalNode('whitecrownedsparrow'),
AnimalNode('lessergoldfinch'),
]),
AnimalNode('insecteaters',[
AnimalNode('hermitthrush'),
Page229
APythonBook
AnimalNode('blackheadedphoebe'),
]),
])
])
deftest():
Tree.show()
test()
Notes:
Werepresentthechildrenofanodeasalist.Eachnode"hasa"listof
children.
Noticethatbecausealistismutable,wedonotusealistconstructor([])in
theinitializerofthemethodheader.Instead,weuseNone,thenconstructan
emptylistinthebodyofthemethodifnecessary.SeesectionOptional
argumentsanddefaultvaluesformoreonthis.
We(recursively)calltheshowmethodforeachnodeinthechildrenlist.
Sinceanodewhichhasnochildren(aleafnode)willhaveanempty
childrenlist,thisprovidesalimitconditionforourrecursion.
3.7.5Classvariables,classmethods,andstaticmethods
Aclassvariableisonewhosesinglevalueissharedbyallinstancesoftheclassand,in
fact,issharedbyallwhohaveaccesstotheclass(object).
"Normal"methodsareinstancemethods.Aninstancemethodreceivestheinstanceasits
firstargument.Ainstancemethodisdefinedbyusingthedefstatementinthebodyofa
classstatement.
Aclassmethodreceivestheclassasitsfirstargument.Aclassmethodisdefinedby
defininganormal/instancemethod,thenusingtheclassmethodbuiltinfunction.For
example:
classASimpleClass(object):
description='asimpleclass'
defshow_class(cls,msg):
print'%s:%s'%(cls.description,msg,)
show_class=classmethod(show_class)
Astaticmethoddoesnotreceiveanythingspecialasitsfirstargument.Astaticmethodis
definedbydefininganormal/instancemethod,thenusingthestaticmethodbuiltin
function.Forexample:
classASimpleClass(object):
description='asimpleclass'
defshow_class(msg):
Page230
APythonBook
print'%s:%s'%(ASimpleClass.description,msg,)
show_class=staticmethod(show_class)
Ineffect,bothclassmethodsandstaticmethodsaredefinedbycreatinganormal
(instance)method,thencreatingawrapperobject(aclassmethodorstaticmethod)using
theclassmethodorstaticmethodbuiltinfunction.
Exercises:
1. Implementaclassthatkeepsarunningtotalofthenumberofinstancescreated.
2. Implementanothersolutiontothesameproblem(aclassthatkeepsarunning
totalofthenumberofinstances),butthistimeuseastaticmethodinsteadofa
classmethod.
Solutions:
1. Weuseaclassvariablenamedinstance_count,ratherthananinstance
variable,tokeeparunningtotalofinstances.Then,weincrementthatvariable
eachtimeaninstanceiscreated:
classCountInstances(object):
instance_count=0
def__init__(self,name='noname'):
self.name=name
CountInstances.instance_count+=1
defshow(self):
print'name:"%s"'%(self.name,)
defshow_instance_count(cls):
print'instancecount:%d'%
(cls.instance_count,)
show_instance_count=
classmethod(show_instance_count)
deftest():
instances=[]
instances.append(CountInstances('apple'))
instances.append(CountInstances('banana'))
instances.append(CountInstances('cherry'))
instances.append(CountInstances())
forinstanceininstances:
instance.show()
CountInstances.show_instance_count()
test()
Notes:
Page231
APythonBook
Whenwerunthisscript,itprintsoutthefollowing:
name:"apple"
name:"banana"
name:"cherry"
name:"noname"
instancecount:4
Thecalltotheclassmethodbuiltinfunctioneffectivelywrapsthe
show_instance_countmethodinaclassmethod,thatis,inamethod
thattakesaclassobjectasitsfirstargumentratherthananinstanceobject.To
readmoreaboutclassmethod,gotoBuiltinFunctions
http://docs.python.org/lib/builtinfuncs.htmlandsearchfor"classmethod".
2. Astaticmethodtakesneitheraninstance(self)noraclassasitsfirst
paramenter.And,staticmethodiscreatedwiththestaticmethod()builtin
function(ratherthanwiththeclassmethod()builtin):
instance_count=0
self.name=name
defshow(self):
defshow_instance_count():
print'instancecount:%d'%(
CountInstances.instance_count,)
show_instance_count=
staticmethod(show_instance_count)
deftest():
instances=[]
instance.show()
test()
3.7.5.1Decoratorsforclassmethodandstaticmethod
Adecoratorenablesustodowhatwedidinthepreviousexamplewithasomewhat
simplersyntax.
Page232
APythonBook
Forsimplecases,thedecoratorsyntaxenablesustodothis:
@functionwrapper
defmethod1(self):
o
o
o
insteadofthis:
defmethod1(self):
o
o
o
method1=functionwrapper(method1)
So,wecanwritethis:
@classmethod
defmethod1(self):
o
o
o
insteadofthis:
defmethod1(self):
o
o
o
method1=classmethod(method1)
Exercises:
1. ImplementtheCountInstancesexampleabove,butuseadecoratorrather
thantheexplicitcalltoclassmethod.
Solutions:
1. Adecoratorisaneasierandcleanerwaytodefineaclassmethod(orastatic
method):
instance_count=0
self.name=name
defshow(self):
@classmethod
Page233
APythonBook
defshow_instance_count(cls):
print'instancecount:%d'%
(cls.instance_count,)
#Notethatthefollowinglinehasbeenreplacedby
#theclassmethoddecorator,above.
#show_instance_count=
classmethod(show_instance_count)
deftest():
instances=[]
instance.show()
test()
3.8AdditionalandAdvancedTopics
3.8.1Decoratorsandhowtoimplementthem
Decoratorscanbeusedto"wrap"afunctionwithanotherfunction.
Whenimplementingadecorator,itishelpfultorememberthatthefollowingdecorator
application:
@dec
deffunc(arg1,arg2):
pass
isequivalentto:
deffunc(arg1,arg2):
pass
func=dec(func)
Therefore,toimplementadecorator,wewriteafunctionthatreturnsafunctionobject,
sincewereplacethevalueoriginallyboundtothefunctionwiththisnewfunctionobject.
Itmaybehelpfultotaketheviewthatwearecreatingafunctionthatisawrapperforthe
originalfunction.
Exercises:
1. Writeadecoratorthatwritesamessagebeforeandafterexecutingafunction.
Solutions:
Page234
APythonBook
1. Afunctionthatcontainsandreturnsaninnerfunctioncanbeusedtowrapa
function:
deftrace(func):
definner(*args,**kwargs):
print'>>'
func(*args,**kwargs)
print'<<'
returninner
@trace
deffunc1(x,y):
print'x:',x,'y:',y
func2((x,y))
@trace
deffunc2(content):
print'content:',content
deftest():
func1('aa','bb')
test()
Notes:
Yourinnerfunctioncanuse*argsand**kwargstoenableittocall
functionswithanynumberofarguments.
3.8.1.1Decoratorswitharguments
Decoratorscanalsotakearguments.
Thefollowingdecoratorwitharguments:
@dec(argA,argB)
deffunc(arg1,arg2):
pass
isequivalentto:
deffunc(arg1,arg2):
pass
func=dec(argA,argB)(func)
Becausethedecorator'sargumentsarepassedtotheresultofcallingthedecoratoronthe
decoratedfunction,youmayfinditusefultoimplementadecoratorwithargumentsusing
afunctioninsideafunctioninsideafunction.
Exercises:
1. Writeandtestadecoratorthattakesoneargument.Thedecoratorprintsa
messagealongwiththevalueoftheargumentbeforeandafterenteringthe
Page235
APythonBook
decoratedfunction.
Solutions:
1. Implementthisdecoratorthattakesargumentswithafunctioncontaininganested
functionwhichinturncontainsanestedfunction:
deftrace(msg):
definner1(func):
definner2(*args,**kwargs):
print'>>[%s]'%(msg,)
retval=func(*args,**kwargs)
print'<<[%s]'%(msg,)
returnretval
returninner2
returninner1
@trace('tracingfunc1')
deffunc1(x,y):
print'x:',x,'y:',y
result=func2((x,y))
returnresult
deffunc2(content):
returncontent*3
deftest():
result=func1('aa','bb')
test()
3.8.1.2Stackeddecorators
Decoratorscanbe"stacked".
Thefollowingstackeddecorators:
@dec2
@dec1
deffunc(arg1,arg2,...):
pass
areequivalentto:
deffunc(arg1,arg2,...):
pass
func=dec2(dec1(func))
Exercises:
1. Implementadecorator(asabove)thattracescallstoadecoratedfunction.Then
Page236
APythonBook
"stack"thatwithanotherdecoratorthatprintsahorizontallineofdashesbefore
andaftercallingthefunction.
2. Modifyyoursolutiontotheaboveexercisesothatthedecoratorthatprintsthe
horizontallinetakesoneargument:acharacter(orcharacters)thatcanberepeated
toproduceahorizontalline/separator.
Solutions:
1. Reuseyourtracingfunctionfromthepreviousexercise,thenwriteasimple
decoratorthatprintsarowofdashes:
deftrace(msg):
definner1(func):
returnretval
returninner2
returninner1
defhorizontal_line(func):
definner(*args,**kwargs):
print''*50
print''*50
returnretval
returninner
deffunc1(x,y):
print'x:',x,'y:',y
result=func2((x,y))
returnresult
@horizontal_line
deffunc2(content):
returncontent*3
deftest():
test()
2. Onceagain,adecoratorwithargumentscanbeimplementedwithafunction
nestedinsideafunctionwhichisnestedinsideafunction.Thisremainsthesame
whetherthedecoratorisusedasastackeddecoratorornot.Hereisasolution:
deftrace(msg):
Page237
APythonBook
definner1(func):
returnretval
returninner2
returninner1
defhorizontal_line(line_chr):
definner1(func):
printline_chr*15
printline_chr*15
returnretval
returninner2
returninner1
deffunc1(x,y):
print'x:',x,'y:',y
result=func2((x,y))
returnresult
@horizontal_line('<**>')
deffunc2(content):
returncontent*3
deftest():
test()
3.8.1.3Morehelpwithdecorators
Thereismoreaboutdecoratorshere:
Pythonsyntaxandsemantics
http://en.wikipedia.org/wiki/Python_syntax_and_semantics#Decoratorsat
Wikipedia.
PythonDecoratorLibraryhttp://wiki.python.org/moin/PythonDecoratorLibrary
atthePythonWikihaslotsofsamplecode.
PEP318DecoratorsforFunctionsandMethods
http://www.python.org/dev/peps/pep0318/istheformalproposaland
specificationforPythondecorators.
Page238
APythonBook
3.8.2Iterables
3.8.2.1AfewpreliminariesonIterables
Definition:iterable(adjective)thatwhichcanbeiteratedover.
Agoodtestofwhethersomethingisiterableiswhetheritcanbeusedinafor:
statement.Forexample,ifwecanwriteforiteminX:,thenXisiterable.Hereis
anothersimpletest:
defisiterable(x):
try:
y=iter(x)
exceptTypeError,exp:
returnFalse
returnTrue
Somekindsofiterables:
ContainersWecaniterateoverlists,tuples,dictionaries,sets,strings,andother
containers.
Somebuiltin(noncontainer)typesExamples:
Atextfileopeninreadmodeisiterable:ititeratesoverthelinesinthefile.
ThexrangetypeSeeXRangeType
http://docs.python.org/lib/typesseqxrange.html.It'susefulwhenyouwanta
largesequenceofintegerstoiterateover.
Instancesofclassesthatobeytheiteratorprotocol.Foradescriptionoftheiterator
protocol,seeIteratorTypeshttp://docs.python.org/lib/typeiter.html.Hint:Type
dir(obj)andlookfor"__iter__"and"next".
GeneratorsAnobjectreturnedbyanyfunctionormethodthatcontainsyield.
Exercises:
1. Implementaclasswhoseinstancesareinterable.Theconstructortakesalistof
URLsasitsargument.Aninstanceofthisclass,wheniteratedover,generatesthe
contentoftheWebpageatthataddress.
Solutions:
1. Weimplementaclassthathas__iter__()andnext()methods:
importurllib
classWebPages(object):
def__init__(self,urls):
self.urls=urls
self.current_index=0
def__iter__(self):
self.current_index=0
returnself
Page239
APythonBook
defnext(self):
ifself.current_index>=len(self.urls):
raiseStopIteration
url=self.urls[self.current_index]
self.current_index+=1
content=f.read()
f.close()
returncontent
deftest():
urls=[
'http://www.python.org',
'http://en.wikipedia.org/',
'http://en.wikipedia.org/wiki/Python_(programming_langu
age)',
]
pages=WebPages(urls)
forpageinpages:
print'length:%d'%(len(page),)
pages=WebPages(urls)
print''*50
page=pages.next()
page=pages.next()
page=pages.next()
page=pages.next()
test()
3.8.2.2Morehelpwithiterables
TheitertoolsmoduleinthePythonstandardlibraryhashelpersforiterators:
http://docs.python.org/library/itertools.html#moduleitertools
3.9ApplicationsandRecipes
3.9.1XMLSAX,minidom,ElementTree,Lxml
Exercises:
1. SAXParseanXMLdocumentwithSAX,thenshowsomeinformation(tag,
attributes,characterdata)foreachelement.
2. MinidomParseanXMLdocumentwithminidom,thenwalktheDOMtree
andshowsomeinformation(tag,attributes,characterdata)foreachelement.
Page240
APythonBook
HereisasampleXMLdocumentthatyoucanuseforinput:
<?xmlversion="1.0"?>
<people>
<personid="1"value="abcd"ratio="3.2">
<name>Alberta</name>
<interest>gardening</interest>
<interest>reading</interest>
<category>5</category>
</person>
<personid="2">
<name>Bernardo</name>
<interest>programming</interest>
<category></category>
<agent>
<firstname>Darren</firstname>
<lastname>Diddly</lastname>
</agent>
</person>
<personid="3"value="efgh">
<name>Charlie</name>
<interest>people</interest>
<interest>cats</interest>
<interest>dogs</interest>
<promoter>
<firstname>David</firstname>
<lastname>Donaldson</lastname>
<client>
<fullname>ArnoldApplebee</fullname>
<refid>10001</refid>
</client>
</promoter>
<promoter>
<firstname>Edward</firstname>
<lastname>Eddleberry</lastname>
<client>
<fullname>ArnoldApplebee</fullname>
<refid>10001</refid>
</client>
</promoter>
</person>
</people>
3. ElementTreeParseanXMLdocumentwithElementTree,thenwalktheDOM
treeandshowsomeinformation(tag,attributes,characterdata)foreachelement.
4. lxmlParseanXMLdocumentwithlxml,thenwalktheDOMtreeandshow
someinformation(tag,attributes,characterdata)foreachelement.
5. ModifydocumentwithElementTreeUseElementTreetoreadadocument,then
modifythetree.Showthecontentsofthetree,andthenwriteoutthemodified
document.
6. XPathlxmlsupportsXPath.UsetheXPathsupportinlxmltoaddresseachof
Page241
APythonBook
thefollowingintheaboveXMLinstancedocument:
Thetextinallthenameelements
Thevaluesofalltheidattributes
Solutions:
1. WecanusetheSAXsupportinthePythonstandardlibrary:
"""
ParseandXMLwithSAX.Displayinfoabouteach
element.
Usage:
pythontest_sax.pyinfilename
Examples:
pythontest_sax.pypeople.xml
"""
importsys
fromxml.saximportmake_parser,handler
classTestHandler(handler.ContentHandler):
def__init__(self):
self.level=0
defshow_with_level(self,value):
print'%s%s'%(''*self.level,value,)
defstartDocument(self):
self.show_with_level('Documentstart')
self.level+=1
defendDocument(self):
self.level=1
self.show_with_level('Documentend')
defstartElement(self,name,attrs):
self.show_with_level('startelementname:
"%s"'%(name,))
self.level+=1
defendElement(self,name):
self.level=1
self.show_with_level('endelementname:
"%s"'%(name,))
defcharacters(self,content):
content=content.strip()
ifcontent:
self.show_with_level('characters:"%s"'%
(content,))
Page242
APythonBook
parser=make_parser()
handler=TestHandler()
parser.setContentHandler(handler)
parser.parse(infilename)
defusage():
print__doc__
sys.exit(1)
defmain():
args=sys.argv[1:]
iflen(args)!=1:
usage()
infilename=args[0]
test(infilename)
main()
2. Theminidommodulecontainsaparse()functionthatenablesustoreadan
XMLdocumentandcreateaDOMtree:
"""ProcessanXMLdocumentwithminidom.
Showthedocumenttree.
Usage:
pythonminidom_walk.py[options]infilename
"""
importsys
fromxml.domimportminidom
defshow_tree(doc):
root=doc.documentElement
show_node(root,0)
defshow_node(node,level):
count=0
ifnode.nodeType==minidom.Node.ELEMENT_NODE:
show_level(level)
print'tag:%s'%(node.nodeName,)
forkeyinnode.attributes.keys():
attr=node.attributes.get(key)
show_level(level+1)
print'attributename:%svalue:"%s"'%
(attr.name,
attr.value,)
if(len(node.childNodes)==1and
node.childNodes[0].nodeType==
Page243
APythonBook
minidom.Node.TEXT_NODE):
show_level(level+1)
print'data:"%s"'%
(node.childNodes[0].data,)
forchildinnode.childNodes:
count+=1
show_node(child,level+1)
returncount
forxinrange(level):
print'',
deftest():
args=sys.argv[1:]
iflen(args)!=1:
print__doc__
sys.exit(1)
docname=args[0]
doc=minidom.parse(docname)
show_tree(doc)
test()
3. ElementTreeenablesustoparseanXMLdocumentandcreateaDOMtree:
"""ProcessanXMLdocumentwithelementtree.
Usage:
pythonelementtree_walk.py[options]infilename
"""
importsys
fromxml.etreeimportElementTreeasetree
defshow_tree(doc):
root=doc.getroot()
show_node(root,0)
show_level(level)
print'tag:%s'%(node.tag,)
forkey,valueinnode.attrib.iteritems():
show_level(level+1)
(key,value,)
ifnode.text:
text=node.text.strip()
Page244
APythonBook
show_level(level+1)
print'text:"%s"'%(node.text,)
ifnode.tail:
tail=node.tail.strip()
show_level(level+1)
print'tail:"%s"'%(tail,)
forchildinnode.getchildren():
forxinrange(level):
print'',
deftest():
args=sys.argv[1:]
iflen(args)!=1:
print__doc__
sys.exit(1)
docname=args[0]
doc=etree.parse(docname)
show_tree(doc)
test()
4. lxmlenablesustoparseanXMLdocumentandcreateaDOMtree.Infact,since
lxmlattemptstomimictheElementTreeAPI,ourcodeisverysimilartothatin
thesolutiontotheElementTreeexercise:
Usage:
pythonlxml_walk.py[options]infilename
"""
#
#Imports:
importsys
fromlxmlimportetree
defshow_tree(doc):
root=doc.getroot()
show_node(root,0)
show_level(level)
Page245
APythonBook
show_level(level+1)
(key,value,)
ifnode.text:
show_level(level+1)
ifnode.tail:
show_level(level+1)
forxinrange(level):
print'',
deftest():
args=sys.argv[1:]
iflen(args)!=1:
print__doc__
sys.exit(1)
docname=args[0]
doc=etree.parse(docname)
show_tree(doc)
test()
5. WecanmodifytheDOMtreeandwriteitouttoanewfile:
Modifythedocumenttreeandthenshowitagain.
WritethemodifiedXMLtreetoanewfile.
Usage:
pythonelementtree_walk.py[options]infilename
outfilename
Options:
Example:
pythonelementtree_walk.pymyxmldoc.xml
myotherxmldoc.xml
"""
importsys
importos
importgetopt
Page246
APythonBook
importtime
#UseElementTree.
fromxml.etreeimportElementTreeasetree
#OruncommenttouseLxml.
#fromlxmlimportetree
defshow_tree(doc):
root=doc.getroot()
show_node(root,0)
show_level(level)
show_level(level+1)
(key,value,)
ifnode.text:
show_level(level+1)
ifnode.tail:
show_level(level+1)
forxinrange(level):
print'',
defmodify_tree(doc,tag,attrname,attrvalue):
root=doc.getroot()
modify_node(root,tag,attrname,attrvalue)
defmodify_node(node,tag,attrname,attrvalue):
ifnode.tag==tag:
node.attrib[attrname]=attrvalue
modify_node(child,tag,attrname,attrvalue)
deftest(indocname,outdocname):
doc=etree.parse(indocname)
show_tree(doc)
print''*50
date=time.ctime()
modify_tree(doc,'person','date',date)
show_tree(doc)
write_output=False
ifos.path.exists(outdocname):
response=raw_input('Outputfile(%s)exists.
Page247
APythonBook
Overwrite?(y/n):'%
outdocname)
ifresponse=='y':
write_output=True
else:
write_output=True
ifwrite_output:
doc.write(outdocname)
print'WrotemodifiedXMLtreeto%s'%
outdocname
else:
print'Didnotwriteoutputfile.'
defusage():
print__doc__
sys.exit(1)
defmain():
args=sys.argv[1:]
try:
opts,args=getopt.getopt(args,'h',['help',
])
except:
usage()
foropt,valinopts:
usage()
iflen(args)!=2:
usage()
indocname=args[0]
outdocname=args[1]
test(indocname,outdocname)
main()
Notes:
TheabovesolutioncontainsanimportstatementforElementTreeand
anotherforlxml.Theoneforlxmliscommentedout,butyoucouldchange
thatifyouwishtouselxmlinsteadofElementTree.Thissolutionwillwork
thesamewaywitheitherElementTreeorlxml.
6. WhenweparseandXMLdocumentwithlxml,eachelement(node)hasan
xpath()method.
#test_xpath.py
fromlxmlimportetree
deftest():
doc=etree.parse('people.xml')
root=doc.getroot()
Page248
APythonBook
printroot.xpath("//name/text()")
printroot.xpath("//@id")
test()
And,whenweruntheabovecode,hereiswhatwesee:
$pythontest_xpath.py
['Alberta','Bernardo','Charlie']
['1','2','3']
FormoreonXPathsee:XMLPathLanguage(XPath)
http://www.w3.org/TR/xpath
3.9.2Relationaldatabaseaccess
YoucanfindinformationaboutdatabaseprogramminginPythonhere:Database
Programminghttp://wiki.python.org/moin/DatabaseProgramming/.
FordatabaseaccessweusethePythonDatabaseAPI.Youcanfindinformationaboutit
here:PythonDatabaseAPISpecificationv2.0
http://www.python.org/dev/peps/pep0249/.
TousethedatabaseAPIwedothefollowing:
1. Usethedatabaseinterfacemoduletocreateaconnectionobject.
2. Usetheconnectionobjecttocreateacursorobject.
3. UsethecursorobjecttoexecuteanSQLquery.
4. Retrieverowsfromthecursorobject,ifneeded.
5. Optionally,commitresultstothedatabase.
6. Closetheconnectionobject.
Ourexamplesusethegadflydatabase,whichiswritteninPython.Ifyouwanttouse
gadfly,youcanfindithere:http://gadfly.sourceforge.net/.gadflyisareasonable
choiceifyouwantaneasytousedatabaseonyourlocalmachine.
Anotherreasonablechoiceforalocaldatabaseissqlite3,whichisinthePython
standardlibrary.HereisadescriptivequotefromtheSQLiteWebsite:
"SQLiteisasoftwarelibrarythatimplementsaselfcontained,
serverless,zeroconfiguration,transactionalSQLdatabaseengine.
SQLiteisthemostwidelydeployedSQLdatabaseengineintheworld.
ThesourcecodeforSQLiteisinthepublicdomain."
Youcanlearnaboutithere:
sqlite3DBAPI2.0interfaceforSQLitedatabases
http://docs.python.org/library/sqlite3.html
SQLitehomepagehttp://www.sqlite.org/
Page249
APythonBook
Thepysqlitewebpagehttp://oss.itsystementwicklung.de/trac/pysqlite/
Ifyouwantorneedtouseanother,enterpriseclassdatabase,forexamplePostgreSQL,
MySQL,Oracle,etc.,youwillneedaninterfacemoduleforyourspecificdatabase.You
canfindinformationaboutdatabaseinterfacemoduleshere:Databaseinterfaces
http://wiki.python.org/moin/DatabaseInterfaces
Excercises:
1. Writeascriptthatretrievesalltherowsinatableandprintseachrow.
2. Writeascriptthatretrievesalltherowsinatable,thenusesthecursorasan
iteratortoprinteachrow.
3. Writeascriptthatusesthecursor'sdescriptionattributetoprintoutthename
andvalueofeachfieldineachrow.
4. Writeascriptthatperformsseveraloftheabovetasks,butusessqlite3instead
ofgadfly.
Solutions:
1. WecanexecuteaSQLqueryandthenretrievealltherowswith
fetchall():
importgadfly
deftest():
connection=gadfly.connect("dbtest1",
"plantsdbdir")
cur=connection.cursor()
cur.execute('select*fromplantsdborderby
p_name')
rows=cur.fetchall()
forrowinrows:
print'2.row:',row
connection.close()
test()
2. Thecursoritselfisaniterator.Ititeratesovertherowsreturnedbyaquery.So,
weexecuteaSQLqueryandthenweusethecursorinafor:statement:
importgadfly
deftest():
connection=gadfly.connect("dbtest1",
"plantsdbdir")
cur=connection.cursor()
p_name')
forrowincur:
printrow
connection.close()
Page250
APythonBook
test()
3. Thedescriptionattributeinthecursorisacontainerthathasanitemdescribing
eachfield:
importgadfly
deftest():
p_name')
forfieldincur.description:
print'field:',field
rows=cur.fetchall()
forrowinrows:
foridx,fieldinenumerate(row):
content='%s:"%s"'%
(cur.description[idx][0],field,)
printcontent,
print
connection.close()
test()
Notes:
ThecommaattheendoftheprintstatementtellsPythonnottoprinta
newline.
Thecur.descriptionisasequencecontaininganitemforeachfield.
Afterthequery,wecanextractadescriptionofeachfield.
4. Thesolutionsusingsqlite3areverysimilartothoseusinggadfly.For
informationonsqlite3,see:sqlite3DBAPI2.0interfaceforSQLite
databaseshttp://docs.python.org/library/sqlite3.html#modulesqlite3.
"""
Performoperationsonsqlite3(plants)database.
Usage:
pythonpy_db_api.pycommand[arg1,...]
Commands:
createcreatenewdatabase.
showshowcontentsofdatabase.
addaddrowtodatabase.Requires3args(name,
descrip,rating).
deleteremoverowfromdatabase.Requires1arg
(name).
Examples:
pythontest1.pycreate
pythontest1.pyshow
pythontest1.pyaddcrenshaw"Themostsucculent
melon"10
pythontest1.pydeletelemon
Page251
APythonBook
"""
importsys
importsqlite3
Values=[
('lemon','brightandyellow','7'),
('peach','succulent','9'),
('banana','smoothandcreamy','8'),
('nectarine','tangyandtasty','9'),
('orange','sweetandtangy','8'),
]
Field_defs=[
'p_namevarchar',
'p_descripvarchar',
#'p_ratinginteger',
'p_ratingvarchar',
]
defcreatedb():
connection=sqlite3.connect('sqlite3plantsdb')
cursor=connection.cursor()
q1="createtableplantsdb(%s)"%(',
'.join(Field_defs))
print'createq1:%s'%q1
cursor.execute(q1)
q1="createindexindex1onplantsdb(p_name)"
cursor.execute(q1)
q1="insertintoplantsdb(p_name,p_descrip,
p_rating)values('%s','%s',%s)"
forspecinValues:
q2=q1%spec
print'q2:"%s"'%q2
cursor.execute(q2)
connection.commit()
showdb1(cursor)
connection.close()
defshowdb():
connection,cursor=opendb()
showdb1(cursor)
connection.close()
defshowdb1(cursor):
cursor.execute("select*fromplantsdborderby
p_name")
hr()
description=cursor.description
Page252
APythonBook
printdescription
print'description:'
forrowdescriptionindescription:
print'%s'%(rowdescription,)
hr()
printrows
print'rows:'
forrowinrows:
print'%s'%(row,)
hr()
print'content:'
forrowinrows:
descrip=row[1]
name=row[0]
rating='%s'%row[2]
print'%s%s%s'%(
name.ljust(12),descrip.ljust(30),
rating.rjust(4),)
defaddtodb(name,descrip,rating):
try:
rating=int(rating)
exceptValueError,exp:
print'Error:ratingmustbeinteger.'
return
cursor.execute("select*fromplantsdbwherep_name
='%s'"%name)
iflen(rows)>0:
ql="updateplantsdbsetp_descrip='%s',
p_rating='%s'wherep_name='%s'"%(
descrip,rating,name,)
print'ql:',ql
cursor.execute(ql)
connection.commit()
print'Updated'
else:
cursor.execute("insertintoplantsdbvalues
('%s','%s','%s')"%(
name,descrip,rating))
connection.commit()
print'Added'
showdb1(cursor)
connection.close()
defdeletefromdb(name):
cursor.execute("select*fromplantsdbwherep_name
='%s'"%name)
Page253
APythonBook
iflen(rows)>0:
cursor.execute("deletefromplantsdbwhere
p_name='%s'"%name)
connection.commit()
print'Plant(%s)deleted.'%name
else:
print'Plant(%s)doesnotexist.'%name
showdb1(cursor)
connection.close()
defopendb():
connection=sqlite3.connect("sqlite3plantsdb")
cursor=connection.cursor()
returnconnection,cursor
defhr():
print''*60
defusage():
print__doc__
sys.exit(1)
defmain():
args=sys.argv[1:]
iflen(args)<1:
usage()
cmd=args[0]
ifcmd=='create':
iflen(args)!=1:
usage()
createdb()
elifcmd=='show':
iflen(args)!=1:
usage()
showdb()
elifcmd=='add':
iflen(args)<4:
usage()
name=args[1]
descrip=args[2]
rating=args[3]
addtodb(name,descrip,rating)
elifcmd=='delete':
iflen(args)<2:
usage()
name=args[1]
deletefromdb(name)
else:
Page254
APythonBook
usage()
main()
3.9.3CSVcommaseparatedvaluefiles
ThereissupportforparsingandgeneratingCSVfilesinthePythonstandardlibrary.See:
csvCSVFileReadingandWriting
http://docs.python.org/library/csv.html#modulecsv.
Exercises:
1. ReadaCSVfileandprintthefieldsincolumns.Hereisasamplefiletouseas
input:
#namedescriptionrating
Lemon,Brightyellowandtart,5
Eggplant,Purpleandshiny,6
Tangerine,Succulent,8
Solutions:
1. UsetheCSVmoduleinthePythonstandardlibrarytoreadaCSVfile:
"""
ReadaCSVfileandprintthecontentsincolumns.
"""
importcsv
infile=open(infilename)
reader=csv.reader(infile)
print'===============
======'
print'NameDescription
Rating'
print'===============
======'
forfieldsinreader:
iflen(fields)==3:
line='%s%s%s'%(fields[0].ljust(20),
fields[1].ljust(40),
fields[2].ljust(4))
printline
infile.close()
defmain():
infilename='csv_report.csv'
test(infilename)
Page255
APythonBook
main()
And,whenrun,hereiswhatitdisplays:
===============
======
NameDescription
Rating
===============
======
LemonBrightyellowandtart
5
EggplantPurpleandshiny
6
TangerineSucculent
8
3.9.4YAMLandPyYAML
YAMLisastructuredtextdatarepresentationformat.Itusesindentationtoindicate
nesting.HereisadescriptionfromtheYAMLWebsite:
"YAML:YAMLAin'tMarkupLanguage
"WhatItIs:YAMLisahumanfriendlydataserializationstandardfor
allprogramminglanguages."
YoucanlearnmoreaboutYAMLandPyYAMLhere:
TheOfficialYAMLWebSitehttp://yaml.org/
PyYAML.orgthehomeofvariousYAMLimplementationsforPython
http://pyyaml.org/
TheYAML1.2specificationhttp://yaml.org/spec/1.2/
Exercises:
1. ReadthefollowingsampleYAMLdocument.Printouttheinformationinit:
american:
BostonRedSox
DetroitTigers
NewYorkYankees
national:
NewYorkMets
ChicagoCubs
AtlantaBraves
2. LoadtheYAMLdatausedinthepreviousexercise,thenmakeamodification(for
example,add"SanFranciscoGiants"totheNationalLeague),thendumpthe
modifieddatatoanewfile.
Solutions:
Page256
APythonBook
1. PrintingoutinformationfromYAMLisas"simple"asprintingoutaPythondata
structure.Inthissolution,weusetheprettyprinterfromthePythonstandard
library:
importyaml
importpprint
deftest():
infile=open('test1.yaml')
data=yaml.load(infile)
infile.close()
pprint.pprint(data)
test()
Wecould,alternatively,readinandthen"load"fromastring:
importyaml
importpprint
deftest():
infile=open('test1.yaml')
data_str=infile.read()
infile.close()
data=yaml.load(data_str)
pprint.pprint(data)
test()
2. TheYAMLdump()functionenablesustodumpdatatoafile:
importyaml
importpprint
deftest():
infile=open('test1.yaml','r')
data=yaml.load(infile)
infile.close()
data['national'].append('SanFranciscoGiants')
outfile=open('test1_new.yaml','w')
yaml.dump(data,outfile)
outfile.close()
test()
Notes:
IfwewanttoproducethestandardYAML"block"styleratherthanthe"flow"
format,thenwecoulduse:
yaml.dump(data,outfile,default_flow_style=False)
Page257
APythonBook
3.9.5Json
HereisaquotefromWikipediaentryforJson:
"JSON(pronounced'Jason'),shortforJavaScriptObjectNotation,isa
lightweightcomputerdatainterchangeformat.Itisatextbased,
humanreadableformatforrepresentingsimpledatastructuresand
associativearrays(calledobjects)."
TheJsontextrepresentationlooksverysimilartoPythonliteralrepresentationofPython
builtindatatypes(forexample,lists,dictionaries,numbers,andstrings).
LearnmoreaboutJsonandPythonsupportforJsonhere:
IntroducingJSONhttp://json.org/
JsonatWikipediahttp://en.wikipedia.org/wiki/Json
pythonjsonhttp://pypi.python.org/pypi/pythonjson
simplejsonhttp://pypi.python.org/pypi/simplejson
Excercises:
1. WriteaPythonscript,usingyourfavoritePythonJsonimplementation(for
examplepythonjsonorsimplejson),thatdumpsthefollowingdata
structuretoafile:
Data={
'rockandroll':
['Elis','TheBeatles','TheRollingStones',],
'country':
['WillieNelson','HankWilliams',]
}
2. WriteaPythonscriptthatreadsJsondatafromafileandloadsitintoPythondata
structures.
Solutions:
1. ThissolutionusessimplejsontostoreaPythondatastructureencodedasJson
inafile:
importsimplejsonasjson
Data={
'rockandroll':
['Elis','TheBeatles','TheRollingStones',],
'country':
['WillieNelson','HankWilliams',]
}
deftest():
fout=open('tmpdata.json','w')
content=json.dumps(Data)
fout.write(content)
Page258
APythonBook
fout.write('\n')
fout.close()
test()
2. Wecanreadthefileintoastring,thendecodeitfromJson:
importsimplejsonasjson
deftest():
fin=open('tmpdata.json','r')
content=fin.read()
fin.close()
data=json.loads(content)
printdata
test()
Notethatyoumaywantsomecontroloverindentation,characterencoding,etc.For
simplejson,youcanlearnaboutthathere:simplejsonJSONencoderanddecoder
http://simplejson.googlecode.com/svn/tags/simplejson2.0.1/docs/index.html.
Page259
APythonBook
4Part4GeneratingPythonBindingsforXML
ThissectiondiscussesaspecificPythontool,specificallyaPythoncodegeneratorthat
generatesPythonbindingsforXMLfiles.
Thus,thissectionwillhelpyouinthefollowingways:
1. Itwillhelpyoulearntouseaspecifictool,namelygenerateDS.py,that
generatesPythoncodetobeusedtoprocessXMLinstancedocumentsofa
particulardocumenttype.
2. Itwillhelpyougainmoreexperiencewithreading,modifyingandusingPython
code.
4.1Introduction
Additionalinformation:
Ifyouplantoworkthroughthistutorial,youmayfindithelpfultolookatthe
samplecodethataccompaniesthistutorial.Youcanfinditinthedistribution
under:
tutorial/
tutorial/Code/
YoucanfindadditionalinformationaboutgenerateDS.pyhere:
http://http://www.davekuhlman.org/#generatedspy
Thatdocumentationisalsoincludedinthedistribution.
generateDS.pygeneratesPythondatastructures(forexample,classdefinitions)from
anXMLschemadocument.ThesedatastructuresrepresenttheelementsinanXML
documentdescribedbytheXMLschema.generateDS.pyalsogeneratesparsersthat
loadanXMLdocumentintothosedatastructures.Inaddition,aseparatefilecontaining
subclasses(stubs)isoptionallygenerated.Theusercanaddmethodstothesubclassesin
ordertoprocessthecontentsofanXMLdocument.
ThegeneratedPythoncodecontains:
AclassdefinitionforeachelementdefinedintheXMLschemadocument.
Amainanddriverfunctionthatcanbeusedtotestthegeneratedcode.
AparserthatwillreadanXMLdocumentwhichsatisfiestheXMLschemafrom
whichtheparserwasgenerated.Theparsercreatesandpopulatesatreestructure
ofinstancesofthegeneratedPythonclasses.
MethodsineachclasstoexporttheinstancebackouttoXML(methodexport)
andtoexporttheinstancetoaliteralrepresentingthePythondatastructure
Page260
APythonBook
(methodexportLiteral).
Eachgeneratedclasscontainsthefollowing:
Aconstructormethod(__init__),withmembervariableinitializers.
Methodswithnamesget_xyzandset_xyzforeachmembervariable"xyz"
or,ifthemembervariableisdefinedwithmaxOccurs="unbounded",
methodswithnamesget_xyz,set_xyz,add_xyz,andinsert_xyz.
(Note:Ifyouusetheuseoldgettersetter,thenyouwillget
methodswithnameslikegetXyzandsetXyz.)
Abuildmethodthatcanbeusedtopopulateaninstanceoftheclassfroma
nodeinanElementTreeorLxmltree.
Anexportmethodthatwillwritetheinstance(andanynestedsubinstances)to
afileobjectasXMLtext.
AnexportLiteralmethodthatwillwritetheinstance(andanynested
subinstances)toafileobjectasPythonliterals(text).
Thegeneratedsubclassfilecontainsone(sub)classdefinitionforeachdata
representationclass.Ifthesubclassfileisused,thentheparsercreatesinstancesofthe
subclasses(insteadofcreatinginstancesofthesuperclasses).Thisenablestheuserto
extendthesubclasseswith"treewalk"methods,forexample,thatprocessthecontentsof
theXMLfile.Theusercanalsogenerateandextendmultiplesubclassfileswhichusea
single,commonsuperclassfile,thusimplementinganumberofdifferentprocessesonthe
sameXMLdocumenttype.
ThisdocumentintroducestheusertogenerateDS.pyandwalkstheuserthrough
severalexamplesthatshowhowtogeneratePythoncodeandhowtousethatgenerated
code.
4.2Generatingthecode
Note:Thesamplefilesusedbelowareunderthetutorial/Code/directory.
Usethefollowingtogethelp:
$generateDS.pyhelp
I'llassumethatgenerateDS.pyisinadirectoryonyourpath.Ifnot,youshoulddo
whateverisnecessarytomakeitaccessibleandexecutable.
HereisasimpleXMLschemadocument:
And,hereishowyoumightgenerateclassesandsubclassesthatprovidedatabindings(a
PythonAPI)forthedefinitionsinthatschema:
$generateDS.pyopeople_api.pyspeople_sub.pypeople.xsd
Page261
APythonBook
And,ifyouwanttoautomaticallyoverwritethegeneratedPythonfiles,usethef
commandlineflagtoforceoverwritewithoutasking:
$generateDS.pyfopeople_api.pyspeople_sub.pypeople.xsd
And,tohardwirethesubclassfilesothatitimportstheAPImodule,usethesuper
commandlinefile.Example:
$generateDS.pyopeople_api.pypeople.xsd
$generateDS.pyspeople_appl1.pysuper=people_apipeople.xsd
Or,dobothatthesametimewiththefollowing:
$generateDS.pyopeople_api.pyspeople_appl1.py
super=people_apipeople.xsd
And,foryoursecondapplication:
$generateDS.pyspeople_appl2.pysuper=people_apipeople.xsd
Ifyoutakealookinsidethesetwo"application"files,youwillseeandimportstatement
likethefollowing:
import???assupermod
Ifyouhadnotusedthesupercommandlineoptionwhengeneratingthe
"application"files,thenyoucouldmodifythatstatementyourself.Thesuper
commandlineoptiondoesthisforyou.
YoucanalsousetheThegraphicalfrontendtoconfigureoptionsandsavethemina
sessionfile,thenusethatsessionfilewithgenerateDS.pytospecifyyourcommand
lineoptions.Forexample:
$generateDS.pysession=test01.session
Youcantestthegeneratedcodebyrunningit.Trysomethinglikethefollowing:
$pythonpeople_api.pypeople.xml
or:
$pythonpeople_appl1.pypeople.xml
Whydoesthiswork?WhycanwerunthegeneratedcodeasaPythonscript?Ifyou
lookatthegeneratedcode,downneartheendofthefileyou'llfindamain()function
thatcallsafunctionnamedparse().Theparsefunctiondoesthefollowing:
1. ParsesyourXMLinstancedocument.
2. UsesyourgeneratedAPItobuildatreeofinstancesofthegeneratedclasses.
3. Usestheexport()methodsinthattreeofinstancestoprintout(export)XML
Page262
APythonBook
thatrepresentsyourgeneratedtreeofinstances.
Exceptforsomeindentation(ignorablewhitespace),thisexportedXMLshouldbethe
sameastheoriginalXMLdocument.So,thatgivesyouareasonablythoroughtestof
yourgeneratedcode.
And,thecodeinthatparse()functiongivesyouahintofhowyoumightbuildyour
ownapplicationspecificcodethatusesthegeneratedAPI(thosegeneratedPython
classes).
4.3UsingthegeneratedcodetoparseandexportanXMLdocument
Nowthatyouhavegeneratedcodeforyourdatamodel,youcantestitbyrunningitasan
application.SupposethatyouhaveanXMLinstancedocumentpeople1.xmlthat
satisfiesyourschema.Thenyoucanparsethatinstancedocumentandexportit(printit
out)withsomethinglikethefollowing:
$pythonpeople_api.pypeople1.xml
And,ifyouhaveusedthesupercommandlineoption,asIhaveabove,toconnect
yoursubclassfilewiththesuperclass(API)file,thenyoucouldusethefollowingtodo
thesamething:
$pythonpeople_appl1.pypeople1.xml
4.4Somecommandlineoptionsyoumightwanttoknow
Youmaywanttomerelyskimthissectionfornow,thenlaterreferbacktoitwhensome
oftheseoptionsareareusedlaterinthistutorial.Also,rememberthatyoucanget
informationaboutmorecommandlineoptionsusedbygenerateDS.pybytyping:
$pythongenerateDS.pyhelp
andbyreadingthedocumentathttp://www.davekuhlman.org/#generatedspy
o
Generatethesuperclassmodule.Thisisthemodulethatcontainstheimplementation
ofeachclassforeachelementtype.So,youcanthinkofthisastheimplementationof
the"databindings"ortheAPIforXMLdocumentsofthetypedefinedbyyourXML
schema.
s
Generatethesubclassmodule.Youmightormightnotneedthese.Ifyouintendto
writesomeapplicationspecificcode,youmightwanttoconsiderstartingwiththese
skeletonclassesandaddyourapplicationcodethere.
Page263
APythonBook
super
Thisoptioninsertsthenameofthesuperclassmoduleintoanimportstatementin
thesubclassfile(generatedwith"s").Ifyouknowthenameofthesuperclassfilein
advance,youcanusethisoptiontoenablethesubclassfiletoimportthesuperclass
moduleautomatically.Ifyoudonotusethisoption,youwillneedtoeditthesubclass
modulewithyourtexteditorandmodifytheimportstatementnearthetop.
rootelement="elementname"
UsethisoptiontotellgenerateDS.pywhichoftheelementsdefinedinyourXM
schemaisthe"root"element.Therootelementistheoutermost(toplevel)element
inXMLinstancedocumentsdefinedbythisschema.Ineffect,thistellsyour
generatedmoduleswhichelementtouseastherootelementwhenparsingand
exportingdocuments.
generateDS.pyattemptstoguesstherootelement,usuallythefirstelement
definedinyourXMLschema.Usethisoptionwhenthatdefaultisnotwhatyouwant.
memberspecs=list|dict
Supposeyouwanttowritesomecodethatcanbegenericallyappliedtoelementsof
differentkinds(elementtypesimplementedbyseveraldifferentgeneratedclasses.If
so,itmightbehelpfultohavealistordictionaryspecifyinginformationabouteach
memberdataitemineachclass.Thisoptiondoesthatbygeneratingalistora
dictionary(withthememberdataitemnameaskey)ineachgeneratedclass.Takea
lookatthegeneratedcodetolearnaboutit.Inparticular,lookatthegeneratedlistor
dictionaryinaclassforanyelementtypeandalsoatthedefinitionoftheclass
_MemberSpecgeneratednearthetopoftheAPImodule.
version
AskgenerateDS.pytotellyouwhatversionitis.Thisishelpfulwhenyouwant
toaskaboutaproblem,forexampleatthegeneratedsusersemaillist
(https://lists.sourceforge.net/lists/listinfo/generatedsusers),andwanttospecifywhich
versionyouareusing.
4.5Thegraphicalfrontend
ThereisalsoapointandclickwaytorungenerateDS.Itenablesyoutospecifythe
optionsneededbygenerateDS.pythroughagraphicalinterface,thentorun
generateDS.pywiththoseoptions.Italso
Youcanrunit,ifyouhaveinstalledgenerateDS,bytypingthefollowingata
commandline:
Page264
APythonBook
$generateds_gui.py
Afterconfiguringoptions,youcansavethoseoptionsina"session"file,whichcanbe
loadedlater.LookundertheFilemenuforsaveandloadcommandsandalsoconsider
usingthe"session"commandlineoption.
AlsonotethatgenerateDS.pyitselfsupportsa"session"commandlineoptionthat
enablesyoutorungenerateDS.pywiththeoptionsthatyouspecifiedandsavedwith
thegraphicalfrontend.
4.6Addingapplicationspecificbehavior
generateDS.pygeneratesPythoncodewhich,withnomodification,willparseand
thenexportanXMLdocumentdefinedbyyourschema.However,youarelikelytowant
togobeyondthat.Inmanysituationsyouwillwanttoconstructacustomapplicationthat
processesyourXMLdocumentsusingthegeneratedcode.
4.6.1Implementingcustomsubclasses
Onestrategyistogenerateasubclassfileandtoaddyourapplicationspecificcodeto
that.Generatethesubclassfilewiththe"s"commandlineflag:
$generateDS.pysmyapp.pypeople.xsd
Nowaddsomeapplicationspecificcodetomyapp.py,forexample,ifyouareusingthe
included"people"samplefiles:
classpeopleTypeSub(supermod.people):
def__init__(self,comments=None,person=None,programmer=None,
python_programmer=None,java_programmer=None):
supermod.people.__init__(self,comments,person,programmer,
python_programmer,
java_programmer)
deffancyexport(self,outfile):
outfile.write('Startingfancyexport')
forpersoninself.get_person():
person.fancyexport(outfile)
supermod.people.subclass=peopleTypeSub
#endclasspeopleTypeSub
classpersonTypeSub(supermod.person):
def__init__(self,vegetable=None,fruit=None,ratio=None,
id=None,value=None,
name=None,interest=None,category=None,agent=None,
promoter=None,
description=None):
supermod.person.__init__(self,vegetable,fruit,ratio,id,
value,
Page265
APythonBook
name,interest,category,agent,promoter,description)
deffancyexport(self,outfile):
outfile.write('Fancypersonexportname:%s'%
self.get_name(),)
supermod.person.subclass=personTypeSub
#endclasspersonTypeSub
4.6.2Usingthegenerated"API"fromyourapplication
Inthisapproachyoumightdothingslikethefollowing:
importyourgeneratedclasses.
Createinstancesofthoseclasses.
Linkthoseinstances,forexampleput"children"insideofaparent,oraddoneor
moreinstancestoaparentthatcancontainalistofobjects(think"maxOccurs"
greaterthan1inyourschema)
GettoknowthegeneratedexportAPIbyinspectingthegeneratedcodeinthesuperclass
file.That'sthefilegeneratedwiththe"o"commandlineflag.
Whattolookfor:
Lookattheargumentstotheconstructor(__init__)tolearnhowtoinitialize
aninstance.
Lookatthe"getters"and"setters"(methodsnamegetxxxandsetxxx,tolearn
howtomodifymembervariables.
Lookforamethodnamedaddxxxformembersthatarelists.Thesecorrespond
tomembersdefinedwithmaxOccurs="n",wheren>1.
Lookatthebuildmethods:build,buildChildren,and
buildAttributes.Thesewillgiveyouinformationabouthowtoconstruct
eachofthemembersofagivenelement/class.
Now,youcanimportyourgeneratedAPImodule,anduseittoconstructandmanipulate
objects.Hereisanexampleusingcodegeneratedwiththe"people"schema:
importsys
importpeople_apiasapi
deftest(names):
people=api.peopleType()
forcount,nameinenumerate(names):
id='%d'%(count+1,)
person=api.personType(name=name,id=id)
people.add_person(person)
people.export(sys.stdout,0)
test(['albert','betsy','charlie'])
Runthisandyoumightseesomethinglikethefollowing:
Page266
APythonBook
$pythontmp.py
<people>
<personid="1">
<name>albert</name>
</person>
<personid="2">
<name>betsy</name>
</person>
<personid="3">
<name>charlie</name>
</person>
</people>
4.6.3Acombinedapproach
Note:Youcanfindexamplesofthecodeinthissectioninthesefiles:
tutorial/Code/upcase_names.py
tutorial/Code/upcase_names_appl.py
Herearetherelevant,modifiedsubclasses(upcase_names_appl.py):
importpeople_apiassupermod
classpeopleTypeSub(supermod.peopleType):
def__init__(self,comments=None,person=None,
specialperson=None,programmer=None,python_programmer=None,
java_programmer=None):
super(peopleTypeSub,self).__init__(comments,person,
specialperson,programmer,python_programmer,java_programmer,)
defupcase_names(self):
forpersoninself.get_person():
person.upcase_names()
supermod.peopleType.subclass=peopleTypeSub
#endclasspeopleTypeSub
classpersonTypeSub(supermod.personType):
def__init__(self,vegetable=None,fruit=None,ratio=None,
id=None,value=None,name=None,interest=None,category=None,
agent=None,promoter=None,description=None,range_=None,
extensiontype_=None):
super(personTypeSub,self).__init__(vegetable,fruit,ratio,
id,value,name,interest,category,agent,promoter,description,
range_,extensiontype_,)
defupcase_names(self):
self.set_name(self.get_name().upper())
supermod.personType.subclass=personTypeSub
#endclasspersonTypeSub
Notes:
Theseclassesweregeneratedwiththe"s"commandlineoption.Theyare
Page267
APythonBook
subclassesofclassesinthemodulepeople_api,whichwasgeneratedwiththe
"o"commandlineoption.
Theonlymodificationtotheskeletonsubclassesistheadditionofthetwo
methodsnamedupcase_names().
InthesubclasspeopleTypeSub,themethodupcase_names()merelywalk
overitsimmediatechildren.
InthesubclasspersonTypeSub,themethodupcase_names()justconverts
thevalueofits"name"membertouppercase.
Hereistheapplicationitself(upcase_names.py):
importsys
importupcase_names_applasappl
defcreate_people(names):
people=appl.peopleTypeSub()
forcount,nameinenumerate(names):
id='%d'%(count+1,)
person=appl.personTypeSub(name=name,id=id)
people.add_person(person)
returnpeople
defmain():
names=['albert','betsy','charlie']
people=create_people(names)
print'Before:'
people.upcase_names()
print''*50
print'After:'
main()
Notes:
Thecreate_people()functioncreatesapeopleTypeSubinstancewith
severalpersonTypeSubinstancesinsideit.
And,whenyourunthisminiapplication,hereiswhatyoumightsee:
$pythonupcase_names.py
Before:
<people>
<personid="1">
<name>albert</name>
</person>
<personid="2">
<name>betsy</name>
</person>
<personid="3">
<name>charlie</name>
Page268
APythonBook
</person>
</people>
After:
<people>
<personid="1">
<name>ALBERT</name>
</person>
<personid="2">
<name>BETSY</name>
</person>
<personid="3">
<name>CHARLIE</name>
</person>
</people>
4.7Specialsituationsanduses
4.7.1Generic,typeindependentprocessing
Therearetimeswhenyouwouldliketoimplementafunctionormethodthatcanperform
operationsonavarietyofmembersandthatneedstypeinformationabouteachmember.
Youcangethelpwiththisbygeneratingyourcodewiththe"memberspecs"command
lineoption.Whenyouusethisoption,generateDS.pyaddalistoradictionary
containinganitemforeachmember.Ifyouwantalist,thenuse"memberspecs=list",
andifyouwantadictionary,withmembernamesaskeys,thenuse
"memberspecs=dict".
HereisanexampleInthisexample,wewalkthedocument/instancetreeandconvert
allstringsimpletypestouppercase.
Hereisaschema(Code/member_specs.xsd):
<xs:schemaxmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:elementname="contactlist"type="contactlistType"/>
<xs:complexTypename="contactlistType">
<xs:sequence>
<xs:elementname="description"type="xs:string"/>
<xs:elementname="contact"type="contactType"
maxOccurs="unbounded"/>
</xs:sequence>
<xs:attributename="locator"type="xs:string"/>
</xs:complexType>
<xs:complexTypename="contactType">
Page269
APythonBook
<xs:sequence>
<xs:elementname="firstname"type="xs:string"/>
<xs:elementname="lastname"type="xs:string"/>
<xs:elementname="interest"type="xs:string"
<xs:elementname="category"type="xs:integer"/>
</xs:sequence>
<xs:attributename="id"type="xs:integer"/>
<xs:attributename="priority"type="xs:float"/>
<xs:attributename="colorcode"type="xs:string"/>
</xs:complexType>
</xs:schema>
4.7.1.1Step1generatethebindings
Wegeneratecodewiththefollowingcommandline:
$generateDS.pyf\
omember_specs_api.py\
smember_specs_upper.py\
super=member_specs_api\
memberspecs=list\
member_specs.xsd
Notes:
Wegeneratethememberspecificationsasalistwiththecommandlineoption
memberspecs=list.
Wegeneratean"application"modulewiththescommandlineoption.We'llput
ourapplicationspecificcodeinmember_specs_upper.py.
4.7.1.2Step2addapplicationspecificcode
And,hereisthesubclassfile(member_specs_upper.py,generatedwiththe"s"
commandlineoption),towhichwehaveaddedabitofcodethatconvertsanystringtype
memberstouppercase.Youcanthinkofthismoduleasaspecial"application"ofthe
generatedclasses.
#
#member_specs_upper.py
#
#
#GeneratedTueNov915:54:472010bygenerateDS.pyversion2.2a.
#
importsys
Page270
APythonBook
importmember_specs_apiassupermod
etree_=None
Verbose_import_=False
(XMLParser_import_none,XMLParser_import_lxml,
XMLParser_import_elementtree
)=range(3)
XMLParser_import_library=None
try:
#lxml
fromlxmlimportetreeasetree_
XMLParser_import_library=XMLParser_import_lxml
ifVerbose_import_:
print("runningwithlxml.etree")
exceptImportError:
try:
#cElementTreefromPython2.5+
importxml.etree.cElementTreeasetree_
XMLParser_import_library=XMLParser_import_elementtree
ifVerbose_import_:
print("runningwithcElementTreeonPython2.5+")
exceptImportError:
try:
#ElementTreefromPython2.5+
importxml.etree.ElementTreeasetree_
XMLParser_import_library=XMLParser_import_elementtree
ifVerbose_import_:
print("runningwithElementTreeonPython2.5+")
exceptImportError:
try:
#normalcElementTreeinstall
importcElementTreeasetree_
XMLParser_import_library=
ifVerbose_import_:
print("runningwithcElementTree")
exceptImportError:
try:
#normalElementTreeinstall
importelementtree.ElementTreeasetree_
XMLParser_import_library=
ifVerbose_import_:
print("runningwithElementTree")
exceptImportError:
raiseImportError("FailedtoimportElementTree
fromanyknownplace")
defparsexml_(*args,**kwargs):
if(XMLParser_import_library==XMLParser_import_lxmland
'parser'notinkwargs):
#UsethelxmlElementTreecompatibleparsersothat,e.g.,
Page271
APythonBook
#weignorecomments.
kwargs['parser']=etree_.ETCompatXMLParser()
doc=etree_.parse(*args,**kwargs)
returndoc
#
#Globals
#
ExternalEncoding='ascii'
#
#Utilityfuntionsneededineachgeneratedclass.
#
defupper_elements(obj):
foriteminobj.member_data_items_:
ifitem.get_data_type()=='xs:string':
name=remap(item.get_name())
val1=getattr(obj,name)
ifisinstance(val1,list):
foridx,val2inenumerate(val1):
val1[idx]=val2.upper()
else:
setattr(obj,name,val1.upper())
defremap(name):
newname=name.replace('','_')
returnnewname
#
#Datarepresentationclasses
#
classcontactlistTypeSub(supermod.contactlistType):
def__init__(self,locator=None,description=None,contact=None):
super(contactlistTypeSub,self).__init__(locator,
description,contact,)
defupper(self):
upper_elements(self)
forchildinself.get_contact():
child.upper()
supermod.contactlistType.subclass=contactlistTypeSub
#endclasscontactlistTypeSub
classcontactTypeSub(supermod.contactType):
def__init__(self,priority=None,color_code=None,id=None,
first_name=None,last_name=None,interest=None,category=None):
super(contactTypeSub,self).__init__(priority,color_code,
id,first_name,last_name,interest,category,)
defupper(self):
Page272
APythonBook
upper_elements(self)
supermod.contactType.subclass=contactTypeSub
#endclasscontactTypeSub
defget_root_tag(node):
tag=supermod.Tag_pattern_.match(node.tag).groups()[1]
rootClass=None
ifhasattr(supermod,tag):
rootClass=getattr(supermod,tag)
returntag,rootClass
defparse(inFilename):
doc=parsexml_(inFilename)
rootNode=doc.getroot()
rootTag,rootClass=get_root_tag(rootNode)
ifrootClassisNone:
rootTag='contactlist'
rootClass=supermod.contactlistType
rootObj=rootClass.factory()
rootObj.build(rootNode)
#EnablePythontocollectthespaceusedbytheDOM.
doc=None
sys.stdout.write('<?xmlversion="1.0"?>\n')
rootObj.export(sys.stdout,0,name_=rootTag,
namespacedef_='')
doc=None
returnrootObj
defparseString(inString):
fromStringIOimportStringIO
doc=parsexml_(StringIO(inString))
ifrootClassisNone:
doc=None
rootObj.export(sys.stdout,0,name_=rootTag,
namespacedef_='')
returnrootObj
defparseLiteral(inFilename):
doc=parsexml_(inFilename)
Page273
APythonBook
ifrootClassisNone:
doc=None
sys.stdout.write('#frommember_specs_apiimport*\n\n')
sys.stdout.write('importmember_specs_apiasmodel_\n\n')
sys.stdout.write('rootObj=model_.contact_list(\n')
rootObj.exportLiteral(sys.stdout,0,name_="contact_list")
sys.stdout.write(')\n')
returnrootObj
USAGE_TEXT="""
Usage:python???.py<infilename>
"""
defusage():
printUSAGE_TEXT
sys.exit(1)
defmain():
args=sys.argv[1:]
iflen(args)!=1:
usage()
infilename=args[0]
root=parse(infilename)
main()
Notes:
Weaddthefunctionsupper_elementsandremapthatweuseineach
generatedclass.
Noticehowthefunctionupper_elementscallsthefunctionremaponlyon
thosememberswhosetypeisxs:string.
Ineachgenerated(sub)class,weaddthemethodsthatwalktheDOMtreeand
applythemethod(upper)thattransformseachxs:stringvalue.
4.7.1.3Step3writeatest/driverharness
Hereisatestdriver(member_specs_test.py)forour(mini)application:
Page274
APythonBook
#
#member_specs_test.py
#
importsys
importmember_specs_apiassupermod
importmember_specs_upper
defprocess(inFilename):
doc=supermod.parsexml_(inFilename)
rootClass=member_specs_upper.contactlistTypeSub
doc=None
rootObj.export(sys.stdout,0,name_="contactlist",
namespacedef_='')
rootObj.upper()
sys.stdout.write(''*60)
sys.stdout.write('\n')
rootObj.export(sys.stdout,0,name_="contactlist",
namespacedef_='')
returnrootObj
USAGE_MSG="""\
Synopsis:
Sampleapplicationusingclassesandsubclassesgeneratedby
generateDS.py
Usage:
pythonmember_specs_test.pyinfilename
"""
defusage():
printUSAGE_MSG
sys.exit(1)
defmain():
args=sys.argv[1:]
iflen(args)!=1:
usage()
infilename=args[0]
process(infilename)
main()
Notes:
Wecopythefunctionparse()fromourgeneratedcodetoserveasamodelfor
Page275
APythonBook
ourfunctionprocess().
AfterparsinganddisplayingtheXMLinstancedocument,wecallmethod
upper()inthegeneratedclasscontactlistTypeSubinordertowalkthe
DOMtreeandtransformeachxs:stringtouppercase.
4.7.1.4Step4runthetestapplication
Wecanusethefollowingcommandlinetorunourapplication:
$pythonmember_specs_test.pymember_specs_data.xml
Whenwerunourapplication,hereistheoutput:
$pythonmember_specs_test.pymember_specs_data.xml
<contactlistlocator="http://www.rexx.com/~dkuhlman">
<description>Mylistofcontacts</description>
<contactpriority="0.050000"colorcode="red"id="1">
<firstname>arlene</firstname>
<lastname>Allen</lastname>
<interest>traveling</interest>
</contact>
</contactlist>
<contactlistlocator="HTTP://WWW.REXX.COM/~DKUHLMAN">
<description>MYLISTOFCONTACTS</description>
<contactpriority="0.050000"colorcode="RED"id="1">
<firstname>ARLENE</firstname>
<lastname>ALLEN</lastname>
<interest>TRAVELING</interest>
</contact>
</contactlist>
Notes:
Theoutputaboveshowsbothbeforeandafterversionofexportingtheparsed
XMLinstancedocument.
4.8Somehints
Thefollowinghintsareofferedforconvenience.Youcandiscoverthemforyourself
rathereasilybyinspectingthegeneratedcode.
4.8.1ChildrendefinedwithmaxOccursgreaterthan1
IfachildelementisdefinedintheXMLschemawithmaxOccurs="unbounded"or
avalueofmaxOccursgreaterthan1,thenaccesstothechildisthroughalist.
Page276
APythonBook
4.8.2Childrendefinedwithsimplenumerictypes
Ifachildelementisdefinedasanumerictypesuchasxs:integer,xs:float,or
xs:doubleorasasimpletypethatis(ultimately)basedonanumerictype,thenthe
valueisstored(inthePythonobject)asaPythondatatype(int,float,etc).
4.8.3Thetypeofanelement'scharactercontent
But,whentheelementitselfisdefinedasmixed="true"ortheelementarestrictionof
andhasasimple(numeric)asabase,thenthevalueOf_instancevariableholdsthe
charactercontentanditisalwaysastring,thatisitisnotconverted.
4.8.4Constructorsandtheirdefaultvalues
Allparameterstotheconstructorsofgeneratedclasseshavedefaultparameters.
Therefore,youcancreatean"empty"instanceofanyelementbycallingtheconstructor
withnoparameters.
Forexample,supposewehavethefollowingXMLschema:
<xs:schemaxmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:elementname="plantlist"type="PlantList"/>
<xs:complexTypename="PlantType">
<xs:sequence>
<xs:elementname="catagory"type="xs:integer"/>
<xs:elementname="fertilizer"type="FertilizerType"
</xs:sequence>
<xs:attributename="identifier"type="xs:string"/>
</xs:complexType>
<xs:complexTypename="FertilizerType">
<xs:sequence>
<xs:elementname="name"type="xs:string"/>
</xs:sequence>
<xs:attributename="id"type="xs:integer"/>
</xs:complexType>
</xs:schema>
And,supposewegenerateamodulewiththefollowingcommandline:
$./generateDS.pyogarden_api.pygarden.xsd
Page277
APythonBook
Then,fortheelementnamedPlantTypeinthegeneratedmodulenamed
garden_api.py,youcancreateaninstanceasfollows:
>>>importgarden_api
>>>plant=garden_api.PlantType()
>>>importsys
>>>plant.export(sys.stdout,0)
<PlantType/>
Page278

A Python Book 2014 PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

A Python Book 2014 PDF

Uploaded by

Copyright:

Available Formats

APythonBook

You might also like