深入 Python 3 中文版

:Python3
PYTHON
Temporamutanturnosetmutamurinillis.
Python3
Python3
PYTHON
Python
ISP
Python3Linux
Python3GNU/Linux
Python2
Python3MacOSXPython2
Python3Microsoft
WindowsPython
Python
LinuxMacOSXPython3
LinuxApplication
Terminal
AccessoriesSystem
MacOSX/Application/Utilities/
Terminal.app
python3
Linux
Python3 Pythonshell
mark@atlantis:~$python3
Python3.0.1+(r301:69556,Apr152009,17:25:52)
[GCC4.3.3]onlinux2
Type"help","copyright","credits"or"license"for
moreinformation.
>>>
exit()Pythonshell
Linux
Python3
mark@manganese:~$python3
bash:python3:commandnotfound
Python
WindowsMacOSX
UbuntuLinux
MICROSOFT WINDOWS
Windows32 64
WindowsXPVistaWindows7Python
Themoreimportantdistinctionis32bitv.
64bit.32
python.org/download/Python
3Windows
Python3.1WindowsWindows
Python3.1WindowsAMD64WindowsAMD64
Python
Python3.x
1.
.msi
WindowsPython
PythonPython
RunPython3
2.
Python3
Python
WindowsAdministrator
WindowsPython
Next
3.
Python
3.1.xC:\Python31\
C:Python
Next[]
4.
Python3
RegisterExtensions
Python(.pyfiles)
Tcl/TkPythonShell
Documentation
docs.python.org
UtilityScripts
2to3.pyPython2
Python3Python2
TestSuitePython
Python
5.
DiskUsage
OKCustomizingPython
Python
6.
Entirefeaturewillbeunavailable.
TestSuite7908KB
Next
7.
8.
Finish
9.
Python3.1
IDLE
PythonShell
[PythonShell]
MAC OS X
Windows
PC)PowerPC
MacPython
python.org/download/Mac
Python3.1MacInstallerDiskImage
3.x2.x
1.
Finder
python3.1.dmg
Build.txt
License.txtReadMe.txt
Python.mpkg
Python.mpkgMacPython
2.
Python
ReadMe.txt
Continue
3.
PythonMac
OSX10.3MacOSX10.2
MacOSX
10.2
Python3
Continue
4.
Python
PythonOpenSourceInitiative
Python
Python
Continue
5.
agree
Python
Agree
6.
Python
CustomizeInstall
7.
PythonFrameworkPython.Python
GUIApplicationsGUIIDLE
PythonShell
UNIXcommandlinetoolsUNIX
python3
PythonDocumentationPython
docs.python.org
ShellprofileupdaterShell
shellTerminal.app
PythonShell
FixsystemPythonPython
MacPython3Python
Python2Python3
Install
8.
/usr/local/bin/
MacPython
OK
9.
10.
Close
11.
/Applications
Python3.1
PythonShellIDLE
IDLEPythonShell
12.
PythonShellPython
PythonShell
[PythonShell]
UBUNTU LINUX
Linux
Ubuntu
LinuxPython3Applications
1.
10,000
Canonical
UbuntuLinuxCanonical
2.
Python3Canonical
3.
Python3
4.
Python3
Python(v3.0)
Python
5.
IDLE(usingPython3.0)
PythonShell
ApplyChanges
6.
IDLE(using
Python3.0)Python(v3.0)
Apply
7.
Canonical
8.
9.
IDLEPythonShellClose
Applications
ProgrammingIDLEPythonShell
10.
PythonShellPython
PythonShell
[PythonShell]
Python3
LinuxBSDSolaris
RedHatLinuxyumFreeBSD
Solarispkgaddfriends
Python3+
Python
PYTHON SHELL
PythonShellPython
PythonShellIDLE
Python
PythonShell
IDLE
PythonShell
>>>1+1
2
>>>PythonShell
PythonShell
1+1PythonShell
Python
ENTERPython
Shell
21+1Python
2
>>>print('Helloworld!')
Helloworld!
Pythonshell
PythonShell
help
>>>help
Typehelp()forinteractivehelp,orhelp(object)for
helpaboutobject.
PythonShellhelpmode
Python
help()
>>>help()
WelcometoPython3.0!Thisistheonlinehelputility.
IfthisisyourfirsttimeusingPython,youshould
definitelycheckout
thetutorialontheInternetat
http://docs.python.org/tutorial/.
Enterthenameofanymodule,keyword,ortopictoget
helponwriting
PythonprogramsandusingPythonmodules.Toquitthis
helputilityand
returntotheinterpreter,justtype"quit".
Togetalistofavailablemodules,keywords,ortopics,
type"modules",
"keywords",or"topics".Eachmodulealsocomeswitha
onelinesummary
ofwhatitdoes;tolistthemoduleswhosesummaries
containagivenword
suchas"spam",type"modulesspam".
help>
>>>help>
Python
help>print
Helponbuiltinfunctionprintinmodulebuiltins:
print(...)
print(value,...,sep='',end='\n',file=sys.stdout)
Printsthevaluestoastream,ortosys.stdoutby
default.
Optionalkeywordarguments:
file:afilelikeobject(stream);defaultstothe
currentsys.stdout.
sep:stringinsertedbetweenvalues,defaulta
space.
end:stringappendedafterthelastvalue,default
anewline.
help>PapayaWhip
noPythondocumentationfoundfor'PapayaWhip'
help>quit
YouarenowleavinghelpandreturningtothePython
interpreter.
Ifyouwanttoaskforhelponaparticularobject
directlyfromthe
interpreter,youcantype"help(object)".Executing
"help('string')"
hasthesameeffectastypingaparticularstringatthe
help>prompt.
>>>
print()print
man
2.
Python
3.
quit()
4.
>>>
PythonShell
1.
PythonShellIDLE,Python
PYTHON
PythonIDLE
IDEs
PythonPython
PythonIDEs
Python3PyDevEclipse
EclipsePythonIDEEclipse
PyDev
ActiveStateKomodoIDE
PythonGNUEmacsPython
PythonShellPython
20019MarkPilgrim
Search
Python3
PYTHON 3
THE MINUS LEVEL

Python
Python2
Python3
Python32to3
2to3Python32to3
Python3
print`x`
chardetPython3
Python2Python3
Python2stringsUnicode
stringsPython3bytesstrings
Unicode
bytesPython3strings
bytes
Strings
Bytesstrings
encoding
HTTPWebhttplib2HTTP
HTTPHTTP
PythonPython3
picklePython2
JSON
chardetPython3
Unicode
Python3Python2
Python3
Python
Python2
Python3

Python
PythonXML
FredrikLundhElementTree
PythonElementTree
XMLXML
Python
Python(PyPI)Python
PyPIPython
20019MarkPilgrim
Search
Python3
PYTHON
Dontburyyourburdeninsaintlysilence.Youhaveaproblem?
Great.Rejoice,divein,andinvestigate.
Ven.HenepolaGunaratana
DIVING IN
Python
[downloadhumansize.py]
SUFFIXES={1000:['KB','MB','GB','TB','PB','EB',
'ZB','YB'],
1024:['KiB','MiB','GiB','TiB','PiB',
'EiB','ZiB','YiB']}
defapproximate_size(size,
a_kilobyte_is_1024_bytes=True):
'''Convertafilesizetohumanreadableform.
Keywordarguments:
sizefilesizeinbytes
a_kilobyte_is_1024_bytesifTrue(default),use
multiplesof1024
ifFalse,usemultiples
of1000
Returns:string
'''
ifsize<0:
raiseValueError('numbermustbenonnegative')
multiple=1024ifa_kilobyte_is_1024_byteselse
1000
forsuffixinSUFFIXES[multiple]:
size/=multiple
ifsize<multiple:
return'{0:.1f}{1}'.format(size,suffix)
raiseValueError('numbertoolarge')
if__name__=='__main__':
print(approximate_size(1000000000000,False))
print(approximate_size(1000000000000))
Windows
c:\home\diveintopython3\examples>c:\python31\python.exe
humansize.py
1.0TB
931.3GiB
MacOSXLinux
you@localhost:~/diveintopython3/examples$python3
humansize.py
1.0TB
931.3GiB
Python
PythonPython
approximate_size()
WindowsExplorerMacOSXFinderLinux
NautilusDolphinThunar
1093TODO
TODO1093bytesTODO1KB
approximate_size()
print(approximate_size(arguments))
approximate_size()
print()
print()
modules
approximate_size()
PythonC++
Pascal
interface/implementation
def
Python
Python
returnPython
None
function
subPython
None
def
approximate_size()size
a_kilobyte_is_1024_bytes
PythonPython
Java
Python
Python
Python
approximate_size()
a_kilobyte_is_1024_bytes
Trueoptional
PythonTrue
approximate_size()
approximate_size()a_kilobyte_is_1024_bytes
FalseFalse
2.
approximate_size()
True
1.
>>>fromhumansizeimportapproximate_size
>>>approximate_size(4000,
a_kilobyte_is_1024_bytes=False)
'4.0KB'
>>>approximate_size(size=4000,
a_kilobyte_is_1024_bytes=False)
'4.0KB'
>>>approximate_size(a_kilobyte_is_1024_bytes=False,
size=4000)
'4.0KB'
>>>approximate_size(a_kilobyte_is_1024_bytes=False,
4000)
File"<stdin>",line1
SyntaxError:nonkeywordargafterkeywordarg
>>>approximate_size(size=4000,False)
File"<stdin>",line1
SyntaxError:nonkeywordargafterkeywordarg
approximate_size()
(size4000
a_kilobyte_is_1024_bytesFalse
2.
approximate_size()size
4000a_kilobyte_is_1024_bytes
False
3.
approximate_size()
a_kilobyte_is_1024_bytesFalse
size4000
1.
5.
size4000
Falsea_kilobyte_is_1024_bytes
Python
4.
Python
docstring
Python
approximate_size()docstring
Keywordarguments:
sizefilesizeinbytes
multiplesof1024
of1000
Returns:string
'''
docstring
docstring
Perl5
qq/.../
docstring
docstring
docstring
Python
docstring
PythonIDE
docstring
docstring
docstring
IMPORT
importPython
sys.path
>>>importsys
>>>sys.path
['',
'/usr/lib/python31.zip',
'/usr/lib/python3.1',
'/usr/lib/python3.1/platlinux2@EXTRAMACHDEPPATH@',
'/usr/lib/python3.1/libdynload',
'/usr/lib/python3.1/distpackages',
'/usr/local/lib/python3.1/distpackages']
>>>sys
<module'sys'(builtin)>
>>>sys.path.insert(0,
'/home/mark/diveintopython3/examples')
>>>sys.path
['/home/mark/diveintopython3/examples',
'',
'/usr/lib/python31.zip',
'/usr/lib/python3.1',
'/usr/lib/python3.1/platlinux2@EXTRAMACHDEPPATH@',
'/usr/lib/python3.1/libdynload',
'/usr/lib/python3.1/distpackages',
'/usr/local/lib/python3.1/distpackages']
sys
sys.path
PythonPython
.py
3.
.pysys
builtinmodulesPython
1.
2.
PythonPythonsys
C
4.
sys.path
Python
Python
Python
5.
sys.path.insert(0,new_path)
sys.pathPython
Python2
3
Python
Python
Python
PythonShell
>>>importhumansize
>>>print(humansize.approximate_size(4096,True))
4.0KiB
>>>print(humansize.approximate_size.__doc__)
Convertafilesizetohumanreadableform.
Keywordarguments:
sizefilesizeinbytes
multiplesof1024
of1000
Returns:string
humansize
Python
PythonShell
2.
approximate_size
humansize.approximate_sizeJava
3.
__doc__
1.
PythonimportPerlrequire
importPython
module.function
requirePerl
module::function
Python
__doc__
docstringsys
path
Python
firstclassobject
Pythonfirstclassobjects
firstclassobjects
first
classobjectsfirstclassobjects
Python
Pythonbeginend
:
ifsize<0:
1000
size/=multiple
ifsize<multiple:
ifforwhile
2.
Pythonifif
trueelse
3.
ifraise
ValueErrorsize<0
4.
1.
5.
for
for
Fortran
Python
Python
Python
C++Java
PythonPython
Python
Python
PythonShell
Python
Shell
ShellPython
JavaPython
bug
try...except
Pythontry...except
raiseJavaC++
try...catchthrow
approximate_size()
size
ifsize<0:
raise
raiseValueError
'numbermustbenonnegative'
Python
traceback
PythonImportError
import
chardet
try..except
try:
importchardet
exceptImportError:
chardet=None
ifchardet
ifchardet:
#dosomething
else:
#continueanyway
ImportError
API
XMLAPI
ElementTreeAPIlxml
xml.etree.ElementTree
Python3
try:
fromlxmlimportetree
exceptImportError:
importxml.etree.ElementTreeasetree
try..except
etreeAPI
etree
if
UNBOUND
approximate_size()
multiple=1024ifa_kilobyte_is_1024_byteselse1000
multiple
PythonPython

NameError
>>>x
Traceback(mostrecentcalllast):
File"<stdin>",line1,in<module>
NameError:name'x'isnotdefined
>>>x=1
>>>x
1
Python
Python
>>>an_integer=1
>>>an_integer
1
>>>AN_INTEGER
NameError:name'AN_INTEGER'isnotdefined
>>>An_Integer
NameError:name'An_Integer'isnotdefined
>>>an_inteGer
NameError:name'an_inteGer'isnotdefined
Python
Python
Python
humansize.py
CPython===
CPython
if
__name__
__name__import
__name__
>>>importhumansize
>>>humansize.__name__
'humansize'
__name____main__Python
iftrue
if
c:\home\diveintopython3>c:\python31\python.exe
humansize.py
1.0TB
931.3GiB
Python
PEP257:Docstringdocstring
docstring
Python
PEP8:Python
PythonPython
20019MarkPilgrim
:Python3
:
Wonderisthefoundationofallphilosophy,inquiryitsprogress,
ignoranceitsend.
MicheldeMontaigne
Python
Python
Python
Python
1.
2.
3.
4.
5.
6.
BooleansTrueFalse
NumbersIntegers12
Floats1.11.2Fractions1/2
2/3ComplexNumber
StringsUnicode
HTML
BytesByteArrays:
JPEG
Lists
Tuples
7.
8.
Sets
Dictionaries
Python
modulefunctionclassmethod
filecompiledcode
name
docstringsClasses
Files
StringsBytes
PythonTrue
False
ifPython
Python
humansize.py
ifsize<0:
size0<size<0
Pythonshell
>>>size=1
>>>size<0
False
>>>size=0
>>>size<0
False
>>>size=1
>>>size<0
True
Python2
True1False0
>>>True+True
2
>>>TrueFalse
1
>>>True*False
0
>>>True/False
ZeroDivisionError:intdivisionormodulobyzero

Python
IntegerFloatingPoint
Python
>>>type(1)
<class'int'>
>>>isinstance(1,int)
True
>>>1+1
2
>>>1+1.0
2.0
>>>type(2.0)
<class'float'>
type()
1int
2.
isinstance()
3.
intintint
4.
intfloatfloatPython
intfloatfloat
1.
>>>float(2)
2.0
>>>int(2.0)
2
>>>int(2.5)
2
>>>int(2.5)
2
>>>1.12345678901234567890
1.1234567890123457
>>>type(1000000000000000)
<class'int'>
1.
2.
3.
4.
5.
6.
float()int
float
int()float
int
int()
int()0
floor
15
Python2intlong
intsys.maxint
232
1Python3
Python2longPEP
237
>>>11/2
5.5
>>>11//2
5
>>>11//2
6
>>>11.0//2
5.0
>>>11**2
121
>>>11%2
1
/int
float
//
2.
1.
3.
4.
5.
6.
//
65
5
//
float
float
**112121
%1125
11
Python2/
Python3/
PEP238
Python
>>>importfractions
>>>x=fractions.Fraction(1,3)
>>>x
Fraction(1,3)
>>>x*2
Fraction(2,3)
>>>fractions.Fraction(6,4)
Fraction(3,2)
>>>fractions.Fraction(0,0)
File"fractions.py",line96,in__new__
raiseZeroDivisionError('Fraction(%s,0)'%
numerator)
ZeroDivisionError:Fraction(0,0)
fractionsfractions
Fraction
3.
Fraction2*(1/3)=(2/3)
Fraction(6/4)=(3/2)
4.
5.
Python
1.
2.
Python
>>>importmath
>>>math.pi
3.1415926535897931
>>>math.sin(math.pi/2)
1.0
>>>math.tan(math.pi/4)
0.99999999999999989
1.
math
2.
mathsin()
cos()tan()asin()
3.
Pythontan(/4)
1.00.99999999999999989
falsetrue
if
falsetrue
>>>defis_it_true(anything):
...ifanything:
...print("yes,it'strue")
...else:
...print("no,it'sfalse")
...
>>>is_it_true(1)
yes,it'strue
>>>is_it_true(1)
yes,it'strue
>>>is_it_true(0)
no,it'sfalse
>>>is_it_true(0.1)
yes,it'strue
>>>is_it_true(0.0)
no,it'sfalse
>>>importfractions
>>>is_it_true(fractions.Fraction(1,2))
yes,it'strue
>>>is_it_true(fractions.Fraction(0,1))
no,it'sfalse
PythonShell
2.
3.
0.0
Python0.0000000000001
0True
4.
n
Fraction(0,n)
1.
Python
PythonPerl5Perl5
@Python
Python
PythonJava
)
ArrayList
>>>a_list=['a','b','mpilgrim','z','example']
>>>a_list
['a','b','mpilgrim','z','example']
>>>a_list[0]
'a'
>>>a_list[4]
'example'
>>>a_list[1]
'example'
>>>a_list[3]
'mpilgrim'
1.
2.
3.
4.
5.
a_list[0]
5a_list[4]
a_list[1]
a_list[n]==
a_list[len(a_list)n]a_list[3]==
a_list[53]==a_list[2]
a_list[0]
>>>a_list
>>>a_list[1:3]
['b','mpilgrim']
>>>a_list[1:1]
['b','mpilgrim','z']
>>>a_list[0:3]
['a','b','mpilgrim']
>>>a_list[:3]
['a','b','mpilgrim']
>>>a_list[3:]
['z','example']
>>>a_list[:]
1.
2.
3.
4.
5.
6.
(??)
a_list[1]
a_list[3]
between.
a_list[0:3]
a_list[0]a_list[3]
a_list[:3]a_list[0:3]0
a_list[3:]a_list[3:5]
a_list[:3]a_list[3:]
,a_list[:n]n
a_list[n:]
a_list
a_list[:]
>>>a_list=['a']
>>>a_list=a_list+[2.0,3]
>>>a_list
['a',2.0,3]
>>>a_list.append(True)
>>>a_list
['a',2.0,3,True]
>>>a_list.extend(['four',''])
>>>a_list
['a',2.0,3,True,'four','']
>>>a_list.insert(0,'')
>>>a_list
['','a',2.0,3,True,'four','']
1.
a_list
2.
append()
3.
4.
extend()
insert()
5.
'':a_list[0]a_list[6]
a_list.insert(0,value)Perl
unshift()
append()extend()
>>>a_list=['a','b','c']
>>>a_list.extend(['d','e','f'])
>>>a_list
['a','b','c','d','e','f']
>>>len(a_list)
6
>>>a_list[1]
'f'
>>>a_list.append(['g','h','i'])
>>>a_list
['a','b','c','d','e','f',['g','h','i']]
>>>len(a_list)
7
>>>a_list[1]
['g','h','i']
1.
extend()
a_list
33
extend6
3.
append()
3append()
4.
6append
77
append
2.
>>>a_list=['a','b','new','mpilgrim','new']
>>>a_list.count('new')
2
>>>'new'ina_list
True
>>>'c'ina_list
False
>>>a_list.index('mpilgrim')
3
>>>a_list.index('new')
2
>>>a_list.index('c')
Traceback(innermostlast):
File"<interactiveinput>",line1,in?ValueError:
list.index(x):xnotinlist
1.
2.
3.
4.
5.
count()
in
count()inTrue
False
index()
0
0
index()
'new'a_list[2]
a_list[4]index()
index()
index()
Python
1
1index()
1
>>>a_list=['a','b','new','mpilgrim','new']
>>>a_list[1]
'b'
>>>dela_list[1]
>>>a_list
['a','new','mpilgrim','new']
>>>a_list[1]
'new'
1.
2.
del
11
>>>a_list.remove('new')
>>>a_list
['a','mpilgrim','new']
>>>a_list
['a','mpilgrim']
ValueError:list.remove(x):xnotinlist
remove()remove()
value
2.
remove()
1.
REMOVING ITEMS FROM A LIST: BONUS

ROUND
pop()pop()
>>>a_list=['a','b','new','mpilgrim']
>>>a_list.pop()
'mpilgrim'
>>>a_list
['a','b','new']
>>>a_list.pop(1)
'b'
>>>a_list
['a','new']
>>>a_list.pop()
'new'
>>>a_list.pop()
'a'
>>>a_list.pop()
IndexError:popfromemptylist
pop()
2.
poppop()
,
3.
pop()
1.
pop()Perl
pop()
Perlshift()
Python
a_list.pop(0)
if
...ifanything:
...else:
...
>>>is_it_true([])
no,it'sfalse
>>>is_it_true(['a'])
yes,it'strue
>>>is_it_true([False])
yes,it'strue
1.
2.
3.
>>>a_tuple=("a","b","mpilgrim","z","example")
>>>a_tuple
('a','b','mpilgrim','z','example')
>>>a_tuple[0]
'a'
>>>a_tuple[1]
'example'
>>>a_tuple[1:3]
('b','mpilgrim')
2.
a_tuple[0]
3.
4.
1.
append()extend()insert()
remove()pop()
#continuedfromthepreviousexample
>>>a_tuple
('a','b','mpilgrim','z','example')
>>>a_tuple.append("new")
File"<interactiveinput>",line1,in?AttributeError:
'tuple'objecthasnoattribute'append'
>>>a_tuple.remove("z")
File"<interactiveinput>",line1,in?AttributeError:
'tuple'objecthasnoattribute'remove'
>>>a_tuple.index("example")
4
>>>"z"ina_tuple
True
append()extend()
2.
remove()pop()
3.
4.
in
1.
assert
tuple()
list()
tuple()list()
if
...ifanything:
...else:
...
>>>is_it_true(())
no,it'sfalse
>>>is_it_true(('a','b'))
yes,it'strue
>>>is_it_true((False,))
yes,it'strue
>>>type((False))
<class'bool'>
>>>type((False,))
<class'tuple'>
4.
Python
1.
2.
3.
Python
>>>v=('a',2,True)
>>>(x,y,z)=v
>>>x
'a'
>>>y
2
>>>z
True
1.
v(x,y,z)
v

range()
>>>(MONDAY,TUESDAY,WEDNESDAY,THURSDAY,FRIDAY,
SATURDAY,SUNDAY)=range(7)
>>>MONDAY
0
>>>TUESDAY
1
>>>SUNDAY
6
range()
range()
MONDAYTUESDAY
WEDNESDAYTHURSDAYFRIDAYSATURDAYSUNDAY
calendar
UNIXcalcalendar
2.
MONDAY0TUESDAY1
1.
Python
os

set
>>>a_set={1}
>>>a_set
{1}
>>>type(a_set)
<class'set'>
>>>a_set={1,2}
>>>a_set
{1,2}
{}
2.
3.
1.
>>>a_list=['a','b','mpilgrim',True,False,42]
>>>a_set=set(a_list)
>>>a_set
{'a',False,'b',True,'mpilgrim',42}
>>>a_list
['a','b','mpilgrim',True,False,42]
set()
set()
2.
3.
1.
>>>a_set=set()
>>>a_set
set()
>>>type(a_set)
<class'set'>
>>>len(a_set)
0
>>>not_sure={}
>>>type(not_sure)
<class'dict'>
set()
{}
3.
4.
5.
Python2
1.
2.
add()update()
>>>a_set={1,2}
>>>a_set.add(4)
>>>a_set
{1,2,4}
>>>len(a_set)
3
>>>a_set.add(1)
>>>a_set
{1,2,4}
>>>len(a_set)
3
1.
add()
2.
3.
4.
>>>a_set={1,2,3}
>>>a_set
{1,2,3}
>>>a_set.update({2,4,6})
>>>a_set
{1,2,3,4,6}
>>>a_set.update({3,6,9},{1,2,3,5,8,13})
>>>a_set
{1,2,3,4,5,6,8,9,13}
>>>a_set.update([10,20,30])
>>>a_set
{1,2,3,4,5,6,8,9,10,13,20,30}
1.
update()
add()
2.
3.
update()
update()
update()
4.
update()
discard()
remove()
>>>a_set={1,3,6,10,15,21,28,36,45}
>>>a_set
{1,3,36,6,10,45,15,21,28}
>>>a_set.discard(10)
>>>a_set
{1,3,36,6,45,15,21,28}
>>>a_set.discard(10)
>>>a_set
{1,3,36,6,45,15,21,28}
>>>a_set.remove(21)
>>>a_set
{1,3,36,6,45,15,28}
>>>a_set.remove(21)
KeyError:21
1.
2.
discard()
discard()
remove()
3.
4.
remove()
KeyError
pop()
>>>a_set={1,3,6,10,15,21,28,36,45}
>>>a_set.pop()
1
>>>a_set.pop()
3
>>>a_set.pop()
36
>>>a_set
{6,10,45,15,21,28}
>>>a_set.clear()
>>>a_set
set()
>>>a_set.pop()
KeyError:'popfromanemptyset'
1.
pop()
clear()
2.
a_set=set()
a_set
3.
KeyError
Python
>>>a_set={2,4,5,9,12,21,30,51,76,127,195}
>>>30ina_set
True
>>>31ina_set
False
>>>b_set={1,2,3,5,6,8,9,12,15,17,18,21}
>>>a_set.union(b_set)
{1,2,195,4,5,6,8,12,76,15,17,18,3,21,30,
51,9,127}
>>>a_set.intersection(b_set)
{9,2,12,5,21}
>>>a_set.difference(b_set)
{195,4,76,51,30,127}
>>>a_set.symmetric_difference(b_set)
{1,3,4,6,8,76,15,17,18,195,127,30,51}
1.
2.
3.
4.
5.
in
union()
intersection()
difference()a_set
b_set
symmetric_difference()
>>>b_set.symmetric_difference(a_set)
{3,1,195,4,6,8,76,15,17,18,51,30,127}
>>>b_set.symmetric_difference(a_set)==
a_set.symmetric_difference(b_set)
True
>>>b_set.union(a_set)==a_set.union(b_set)
True
>>>b_set.intersection(a_set)==
a_set.intersection(b_set)
True
>>>b_set.difference(a_set)==a_set.difference(b_set)
False
1.
2.
3.
4.
5.
a_setb_set b_seta_set
PythonShell
Union
Intersection
Difference
>>>a_set={1,2,3}
>>>b_set={1,2,3,4}
>>>a_set.issubset(b_set)
True
>>>b_set.issuperset(a_set)
True
>>>a_set.add(5)
>>>a_set.issubset(b_set)
False
>>>b_set.issuperset(a_set)
False
1.
a_setb_seta_setb_set
2.
b_seta_seta_set
b_set
3.
a_setb_set
False
if
...ifanything:
...else:
...
>>>is_it_true(set())
no,it'sfalse
>>>is_it_true({'a'})
yes,it'strue
>>>is_it_true({False})
yes,it'strue
1.
2.
3.
Python
PythonPerl5hash[]
Perl5%
PythonPython
>>>a_dict={'server':'db.diveintopython3.org',
'database':'mysql'}
>>>a_dict
{'server': 'db.diveintopython3.org', 'database': 'mysql'}
>>>a_dict['server']
'db.diveintopython3.org'
>>>a_dict['database']
'mysql'
>>>a_dict['db.diveintopython3.org']
KeyError:'db.diveintopython3.org'
a_dict
'server'a_dict['server']
2.
'database'a_dict['database']
3.
'mysql'
1.
4.
a_dict['server']'db.diveintopython3.org'
a_dict['db.diveintopython3.org']
>>>a_dict
{'server': 'db.diveintopython3.org', 'database': 'mysql'}
>>>a_dict['database']='blog'
>>>a_dict
{'server':'db.diveintopython3.org','database':'blog'}
>>>a_dict['user']='mark'
>>>a_dict
{'server':'db.diveintopython3.org','user':'mark',
'database':'blog'}
>>>a_dict['user']='dora'
>>>a_dict
{'server':'db.diveintopython3.org','user':'dora',
'database':'blog'}
>>>a_dict['User']='mark'
>>>a_dict
{'User':'mark','server':'db.diveintopython3.org',
'user':'dora','database':'blog'}
1.
2.
3.
4.
5.
'user''mark'
user"mark"
U"User"
Python
yourfirstPythonprogram
'ZB','YB'],
'EiB','ZiB','YiB']}
shell
>>>SUFFIXES={1000:['KB','MB','GB','TB','PB',
'EB','ZB','YB'],
...1024:['KiB','MiB','GiB','TiB','PiB',
'EiB','ZiB','YiB']}
>>>len(SUFFIXES)
2
>>>1000inSUFFIXES
True
>>>SUFFIXES[1000]
['KB','MB','GB','TB','PB','EB','ZB','YB']
>>>SUFFIXES[1024]
['KiB','MiB','GiB','TiB','PiB','EiB','ZiB','YiB']
>>>SUFFIXES[1000][3]
'TB'
len()
in
1000SUFFIXES8
3.
8
4.
1024SUFFIXES8
5.
SUFFIXES[1000]0
1.
2.
if
...ifanything:
...else:
...
>>>is_it_true({})
no,it'sfalse
>>>is_it_true({'a':1})
yes,it'strue
1.
2.
NONE
NonePythonNone
FalseNone0NoneNone
NoneFalse
NoneNoneType
NoneNoneType
None
>>>type(None)
<class'NoneType'>
>>>None==False
False
>>>None==0
False
>>>None==''
False
>>>None==None
True
>>>x=None
>>>x==None
True
>>>y=None
>>>x==y
True
NONE
NonenotNone
...ifanything:
...else:
...
>>>is_it_true(None)
no,it'sfalse
>>>is_it_true(notNone)
yes,it'strue
fractions
math
PEP237:
PEP238:
20019MarkPilgrim
Search
:Python3
Ourimaginationisstretchedtotheutmost,not,asinfiction,to
imaginethingswhicharenotreallythere,butjusttocomprehend
thosethingswhichare.
RichardFeynman
Python3os(operating
system)os()
Python
API

PythonPython
Shell:
1.
2.
3.
examples
ImportError?Python
examples
:
1.
2.
examples
examples
Python
PythonShell
Python Web PythonCGI
os
>>>importos
>>>print(os.getcwd())
C:\Python31
>>>os.chdir('/Users/pilgrim/diveintopython3/examples')
C:\Users\pilgrim\diveintopython3\examples
1.
2.
os Python;
os.getcwd()
PythonShell PythonShell
Windows
Python ;c:\Python31
PythonShell python3
3.
os.chdir()
4.
os.chdir() Windows
Linux () Python
os.pathos.path
.
>>>importos
>>>
print(os.path.join('/Users/pilgrim/diveintopython3/examp
les/','humansize.py'))
/Users/pilgrim/diveintopython3/examples/humansize.py
>>>
print(os.path.join('/Users/pilgrim/diveintopython3/examp
les','humansize.py'))
/Users/pilgrim/diveintopython3/examples\humansize.py
>>>print(os.path.expanduser('~'))
c:\Users\pilgrim
>>>print(os.path.join(os.path.expanduser('~'),
'diveintopython3','examples','humansize.py'))
c:\Users\pilgrim\diveintopython3\examples\humansize.py
1.
os.path.join()
.
2.
join
Windows
Linux
MacOSX .
Python
os.path.expanduser()~
3.
Home Home
( LinuxMacOSX Windows)
os.path.join()
4.
Home
os.path.join()
addSlashIfNecessary()
Python
os.path
>>>pathname=
'/Users/pilgrim/diveintopython3/examples/humansize.py'
>>>os.path.split(pathname)
('/Users/pilgrim/diveintopython3/examples',
'humansize.py')
>>>(dirname,filename)=os.path.split(pathname)
>>>dirname
'/Users/pilgrim/diveintopython3/examples'
>>>filename
'humansize.py'
>>>(shortname,extension)=os.path.splitext(filename)
>>>shortname
'humansize'
>>>extension
'.py'
1.
2.
split
?
os.path.split()split
3.
dirname os.path.split()
4.
filename os.path.split()
os.path os.path.splitext()
5.

glob Python
glob shell
>>>os.chdir('/Users/pilgrim/diveintopython3/')
>>>importglob
>>>glob.glob('examples/*.xml')
['examples\\feedbroken.xml',
'examples\\feedns0.xml',
'examples\\feed.xml']
>>>os.chdir('examples/')
>>>glob.glob('*test*.py')
['alphameticstest.py',
'pluraltest1.py',
'pluraltest2.py',
'pluraltest3.py',
'pluraltest4.py',
'pluraltest5.py',
'pluraltest6.py',
'romantest1.py',
'romantest10.py',
'romantest2.py',
'romantest3.py',
'romantest4.py',
'romantest5.py',
'romantest6.py',
'romantest7.py',
'romantest8.py',
'romantest9.py']
1.
glob
*.xml
examples .xml
2.
examplesos.chdir()
.
3.
glob
.py test
:
Python API
>>>importos
c:\Users\pilgrim\diveintopython3\examples
>>>metadata=os.stat('feed.xml')
>>>metadata.st_mtime
1247520344.9537716
>>>importtime
>>>time.localtime(metadata.st_mtime)
time.struct_time(tm_year=2009,tm_mon=7,tm_mday=13,
tm_hour=17,
tm_min=25,tm_sec=44,tm_wday=0,tm_yday=194,tm_isdst=1)
examples
feed.xml examples os.stat()
st_mtime(
3.
1970 1 1
)
time Python
4.
time.localtime()
5.
(os.stat() st_mtime)
2009 7 13 5:25
1.
2.
>>>metadata.st_size
3070
>>>importhumansize
>>>humansize.approximate_size(metadata.st_size)
'3.0KiB'
1.
os.stat() st_size
feed.xml3070
2.
st_sizeapproximate_size()
glob.glob()
'examples\feed.xml'
'romantest1.py'
os.path.realpath()
>>>importos
c:\Users\pilgrim\diveintopython3\examples
>>>print(os.path.realpath('feed.xml'))
c:\Users\pilgrim\diveintopython3\examples\feed.xml
Python
.
>>>a_list=[1,9,8,4]
>>>[elem*2forelemina_list]
[2,18,16,8]
>>>a_list
[1,9,8,4]
>>>a_list=[elem*2forelemina_list]
>>>a_list
[2,18,16,8]
a_list
Python a_list
elem Python elem*2
2.
3.
Python
1.
Python os
>>>importos,glob
>>>glob.glob('*.xml')
['feedbroken.xml','feedns0.xml','feed.xml']
>>>[os.path.realpath(f)forfinglob.glob('*.xml')]
['c:\\Users\\pilgrim\\diveintopython3\\examples\\feed
broken.xml',
'c:\\Users\\pilgrim\\diveintopython3\\examples\\feed
ns0.xml',
'c:\\Users\\pilgrim\\diveintopython3\\examples\\feed.xml
']
1.
2.
.xml
.xml
>>>importos,glob
>>>[fforfinglob.glob('*.py')ifos.stat(f).st_size
>6000]
['pluraltest6.py',
'romantest10.py',
'romantest6.py',
'romantest7.py',
'romantest8.py',
'romantest9.py']
1.
if
if
True
.pyif
6000 6
>>>importos,glob
>>>[(os.stat(f).st_size,os.path.realpath(f))forfin
glob.glob('*.xml')]
[(3074,
broken.xml'),
(3386,
ns0.xml'),
(3070,
'c:\\Users\\pilgrim\\diveintopython3\\examples\\feed.xml
')]
>>>importhumansize
>>>[(humansize.approximate_size(os.stat(f).st_size),f)
forfinglob.glob('*.xml')]
[('3.0KiB','feedbroken.xml'),
('3.3KiB','feedns0.xml'),
('3.0KiB','feed.xml')]
.xml
( os.stat())
( os.path.realpath())
2.
.xml
approximate_size()
1.
>>>importos,glob
>>>metadata=[(f,os.stat(f))forfin
glob.glob('*test*.py')]
>>>metadata[0]
('alphameticstest.py',nt.stat_result(st_mode=33206,
st_ino=0,st_dev=0,
st_nlink=0,st_uid=0,st_gid=0,st_size=2509,
st_atime=1247520344,
st_mtime=1247520344,st_ctime=1247520344))
>>>metadata_dict={f:os.stat(f)forfin
glob.glob('*test*.py')}
>>>type(metadata_dict)
<class'dict'>
>>>list(metadata_dict.keys())
['romantest8.py','pluraltest1.py','pluraltest2.py',
'pluraltest5.py',
'pluraltest6.py','romantest7.py','romantest10.py',
'romantest4.py',
'romantest9.py','pluraltest3.py','romantest1.py',
'romantest2.py',
'romantest3.py','romantest5.py','romantest6.py',
'alphameticstest.py',
'pluraltest4.py']
>>>metadata_dict['alphameticstest.py'].st_size
2509
1.
2.
3.
4.
5.
6.
;test
.py(
os.stat())
( f);
( os.stat(f))
glob.glob('*test*.py')
os.stat()
st_size alphameticstest.py
2509
if
>>>importos,glob,humansize
>>>metadata_dict={f:os.stat(f)forfin
glob.glob('*')}
>>>humansize_dict=
{os.path.splitext(f)[0]:humansize.approximate_size(meta.
st_size)\
...forf,metainmetadata_dict.items()if
meta.st_size>6000}
>>>list(humansize_dict.keys())
['romantest9','romantest8','romantest7','romantest6',
'romantest10','pluraltest6']
>>>humansize_dict['romantest9']
'6.5KiB'
(glob.glob('*')) os.stat(f)
2.
6000
(ifmeta.st_size>6000)
(os.path.splitext(f)[0])
(humansize.approximate_size(meta.st_size))
3.
6
6
4.
approximate_size()
1.
>>>a_dict={'a':1,'b':2,'c':3}
>>>{value:keyforkey,valueina_dict.items()}
{1:'a',2:'b',3:'c'}
,
:
>>>a_set=set(range(10))
>>>a_set
{0,1,2,3,4,5,6,7,8,9}
>>>{x**2forxina_set}
{0,1,4,81,64,9,16,49,25,36}
>>>{xforxina_setifx%2==0}
{0,8,2,4,6}
>>>{2**xforxinrange(10)}
{32,1,2,4,8,64,128,256,16,512}
09
2.
if
3.
;
1.
osmodule
osPortableaccesstooperatingsystemspecificfeatures
os.pathmodule
os.pathPlatformindependentmanipulationoffilenames
globmodule
globFilenamepatternmatching
timemodule
timeFunctionsformanipulatingclocktime
Listcomprehensions
Nestedlistcomprehensions
Loopingtechniques
20019MarkPilgrim
Search
:HomeDiveIntoPython3
:
Imtellingyouthiscauseyoureoneofmyfriends.
Myalphabetstartswhereyouralphabetends!
Dr.Seuss,OnBeyondZebra!
Bougainville
Rotokas 12 A,E,G,I,K,O,P,R,S,T,U,
V
26
52
!@#$%&
(bit)(byte)
(characterencoding)
(')
(0255)
ASCII
0127 65 A97
a&c 128
2
7
(diacriticalmarks)
CP1252windows1252
CP1252 ASCII
0127 CP1252 (nwitha
tildeoverit,241)(uwithtwodotsoverit,252)
128255
255

(0
255)
ASCII
&c
email web
Ohno!
email
(modes)
koi8r 241 Mac
Greek 241
UNICODE
Unicode
Unicode 4
(ideograph)
65535 2
(etymological)
U+0041 'A'
'A'
4
1
2
Unicode 1 4
UTF82 32 =4 UTF32
Unicode 4
N N
4Nth
4
Unicode
65535 Unicode
UTF16( 16 =2 )UTF16 065535
2
(astralplane) 65535 Unicode
UTF16
UTF32 2
65535 UTF32 4
UTF32 UTF16
U+4E2D UTF16 4E2D 2D

4E(bigendian)
(littleendian) UTF32
(byteorder)
4E
2D U+4E2D U+2D4E
Unicode
(ByteOrderMark)
UTF16U+FEFFFFFE
UTF16
(oneway)FEFF
UTF16 ASCII
ASCII
(element)
(attribute) Nth
N
ASCII
+(concatenate)
humansize.py
[downloadhumansize.py]
'ZB','YB'],
'EiB','ZiB','YiB']}
Keywordarguments:
sizefilesizeinbytes
multiplesof1024
of1000
Returns:string
'''
ifsize<0:
1000
size/=multiple
ifsize<multiple:
1.
2.
'KB','MB','GB'
(docstring)
3
3.
3
4.
5.
Python3 (format)
(placeholder)
>>>username='mark'
>>>password='PapayaWhip'
>>>"{0}'spasswordis{1}".format(username,password)
"mark'spasswordisPapayaWhip"
1.
2.
PapayaWhip
{0}{1}
(replacementfield) format()
format()
{0} username
{1}password&c
format()
>>>importhumansize
>>>si_suffixes=humansize.SUFFIXES[1000]
>>>si_suffixes
['KB','MB','GB','TB','PB','EB','ZB','YB']
>>>'1000{0[0]}=1{0[1]}'.format(si_suffixes)
'1000KB=1MB'
1.
humansize
(SI, Systme
International) 1000
2.
{0}
format() si_suffixes
si_suffixes {0[0]} si_suffixes
'KB'{0[1]}
'MB' 1000
'1000KB=1MB'
{0} format() 1 {1} 2
Python
(compound
fieldnames)
>>>importhumansize
>>>importsys
>>>'1MB=
1000{0.modules[humansize].SUFFIXES[1000][0]}'.format(sys)
'1MB=1000KB'
sys Python
format()
{0} sys
sys.modulesisadictionaryofallthemodules
thathavebeenimportedinthisPythoninstance.
Thekeysarethemodulenamesasstrings;the
valuesarethemoduleobjectsthemselves.Sothe
replacementfield{0.modules}referstothe
dictionaryofimportedmodules.sys.modules
Python
{0.modules}
sys.modules['humansize']humansize
{0.modules[humansize]}humansize
Python
sys.modules
'humansize'
humansizePEP3101
sys.modules['humansize'].SUFFIXES humansize
{0.modules[humansize].SUFFIXES}
sys.modules['humansize'].SUFFIXES[1000] SI
['KB','MB','GB','TB','PB','EB',
'ZB','YB']
{0.modules[humansize].SUFFIXES[1000]}
sys.modules['humansize'].SUFFIXES[1000][0] SI
'KB'
{0.modules[humansize].SUFFIXES[1000][0]}
KB
humansize.py
ifsize<multiple:
{1} format() suffix
{0:.1f}{0}
:.1f
(formatspecifier)
C printf()
(zero
padding)(spacepadding)
(alignstrings) 10
16
(:).1
f
10
size 698.24suffix 'GB''698.2
GB' 698.24 'GB'
>>>'{0:.1f}{1}'.format(698.24,'GB')
'698.2GB'
Python
>>>s='''Finishedfilesarethere
...sultofyearsofscientif
...icstudycombinedwiththe
...experienceofyears.'''
>>>s.splitlines()
['Finishedfilesarethere',
'sultofyearsofscientif',
'icstudycombinedwiththe',
'experienceofyears.']
>>>print(s.lower())
finishedfilesarethere
sultofyearsofscientif
icstudycombinedwiththe
experienceofyears.
>>>s.lower().count('f')
6
1.
Python shell
(multiline)
ENTER Pythonshell
ENTER
2.
splitlines()
3.
lower()upper()
4.
count()
6 f
key1=value1&key2=value2
{key1:value1,key2:value2}
>>>query=
'user=pilgrim&database=master&password=PapayaWhip'
>>>a_list=query.split('&')
>>>a_list
['user=pilgrim','database=master',
'password=PapayaWhip']
>>>a_list_of_lists=[v.split('=',1)forvina_list]
>>>a_list_of_lists
[['user','pilgrim'],['database','master'],
['password','PapayaWhip']]
>>>a_dict=dict(a_list_of_lists)
>>>a_dict
{'password':'PapayaWhip','user':'pilgrim','database':
'master'}
1.
split()
&
2.
'key=value=foo'.split('=')
['key','value','foo']
3.
dict() Python
(listoflists)
URL
(queryparameters)
URL
URL
urllib.parse.parse_qs()
(slice)
(slicinglists)
>>>a_string='Myalphabetstartswhereyouralphabet
ends.'
>>>a_string[3:11]
'alphabet'
>>>a_string[3:3]
'alphabetstartswhereyouralphabeten'
>>>a_string[0:2]
'My'
>>>a_string[:18]
'Myalphabetstarts'
>>>a_string[18:]
'whereyouralphabetends.'
1.
slice
2.
3.
0 a_string[0:2]
a_string[0]
a_string[2]
4.
Python 0
a_string[:18] a_string[0:18] 0
Python
5.
2
a_string[18:] a_string[18:44]
44
44
a_string[:18] 18 a_string[18:]
18 a_string[:n]
n a_string[n:]
STRING VS. BYTES

(immutable)
Unicode string 0 255
bytes
>>>by=b'abcd\x65'
>>>by
b'abcde'
>>>type(by)
<class'bytes'>
>>>len(by)
5
>>>by+=b'\xff'
>>>by
b'abcde\xff'
>>>len(by)
6
>>>by[0]
97
>>>by[0]=102
TypeError:'bytes'objectdoesnotsupportitem
assignment
1.
byte b'' bytes byte

ASCII \x00 \xff
16
2.
bytes bytes
3.
len()
bytes
4.
+ bytes
bytes
5.
5 1 bytes 6
bytes
6.
bytes
bytes 0
255
7.
bytes
()bytes
bytearray
>>>by=b'abcd\x65'
>>>barr=bytearray(by)
>>>barr
bytearray(b'abcde')
>>>len(barr)
5
>>>barr[0]=102
>>>barr
bytearray(b'fbcde')
1.
bytearray() bytes
bytearray
2.
bytes bytearray
3.
bytearray
0255
bytes strings
>>>by=b'd'
>>>s='abcde'
>>>by+s
TypeError:can'tconcatbytestostr
>>>s.count(by)
TypeError:Can'tconvert'bytes'objecttostr
implicitly
>>>s.count(by.decode('ascii'))
1
1.
bytes
2.
bytes
bytes
Python3 bytes
3.
bytes
bytes
decode()
bytes
encode()
bytes
ASCII
Unicode
>>>a_string='Python'
>>>len(a_string)
9
>>>by=a_string.encode('utf8')
>>>by
b'\xe6\xb7\xb1\xe5\x85\xa5Python'
>>>len(by)
13
>>>by=a_string.encode('gb18030')
>>>by
b'\xc9\xee\xc8\xebPython'
>>>len(by)
11
>>>by=a_string.encode('big5')
>>>by
b'\xb2`\xa4JPython'
>>>len(by)
11
>>>roundtrip=by.decode('big5')
>>>roundtrip
'Python'
>>>a_string==roundtrip
True
1.
a_string 9
2.
by bytes 13 a_string
UTF8
3.
bybytes 11 a_string
GB18030
4.
bybytes 11
a_stringBig5
5.
roundtrip 9 by
Big5
roundtrip a_string
PYTHON
Python3 .py UTF

8
Python2 .py
ASCIIPython3 UTF8
Python
(encoding
declaration).py windows1252
#*coding:windows1252*
UNIXhashbang
#!/usr/bin/python3
#*coding:windows1252*
PEP263:Python

Python Unicode
PythonUnicodeHOWTO
Python3 :vs.Unicode
vs.8bit
Unicode
Unicode
Unicode
(characterstring)
vs.
XML
HTML
string
PEP3101:
20019MarkPilgrim
Search
:Python3
UpdatedOctober7,2009Difficultylevel:
Somepeople,whenconfrontedwithaproblem,thinkIknow,Ill
useregularexpressions.Nowtheyhavetwoproblems.
JamieZawinski
python
index()find()split()count()
replace()
index()
s.lower()
s.upper()
replace() split()
string
if
split()join()
perljavascript phppython
re
>>>s='100NORTHMAINROAD'
>>>s.replace('ROAD','RD.')
'100NORTHMAINRD.'
>>>s='100NORTHBROADROAD'
>>>s.replace('ROAD','RD.')
'100NORTHBRD.RD.'
>>>s[:4]+s[4:].replace('ROAD','RD.')
'100NORTHBROADRD.'
>>>importre
>>>re.sub('ROAD$','RD.',s)
'100NORTHBROADRD.'
1.
2.
3.
4.
5.
ROAD.RD
string replace()
ROAD
s.replace()
ROADROAD
ROADrepalce()
ROAD
ROAD
s[4:]s[:4]
.STSTREET s[6:]
STREET s[:6]
BUG
python
re
ROAD$ROAD
$
^ re.sub()
s ROAD$
RD.ROAD
BROADROAD
^.$
ROAD
BROAD
BROAD 4
>>>s='100BROAD'
>>>re.sub('ROAD$','RD.',s)
'100BRD.'
>>>re.sub('\\bROAD$','RD.',s)
'100BROAD'
>>>re.sub(r'\bROAD$','RD.',s)
'100BROAD'
>>>s='100BROADROADAPT.3'
>>>re.sub(r'\bROAD$','RD.',s)
'100BROADROADAPT.3'
>>>re.sub(r'\bROAD\b','RD.',s)
'100BROADRD.APT3'
ROAD
\b
python \
\\
perl python
perl
bug
bug
2.
\
r python
\t r\t
\ t
3.
ROAD
1.
'ROAD'
re.sub()
4.
$
\b
ROAD
CopyrightMCMXLVICopyright
1946
I=1
V=5
X=10
L=50
C=100
D=500
M=1000
I 1II 2III
3VI 651VII 7VIII
8
10 IXC M
4 5
IIII 4 IV 5 140
XL 50 1041 XLI42 XLII43 XLIII
44 XLIV 50 10 5 1
9 10 8 VIII
9 IX 10 1 VIIII 4
90 XC900 CM
5 10 X
VV 100 C LL
DC
600 CD 400 500 100CI
101IC 100 1
XCIX 100 10 10 1
1000
M
>>>importre
>>>pattern='^M?M?M?$'
>>>re.search(pattern,'M')
<_sre.SRE_Matchobjectat0106FB58>
>>>re.search(pattern,'MM')
<_sre.SRE_Matchobjectat0106C290>
>>>re.search(pattern,'MMM')
<_sre.SRE_Matchobjectat0106AA38>
>>>re.search(pattern,'MMMM')
>>>re.search(pattern,'')
<_sre.SRE_Matchobjectat0106F4A8>
1.
2.
3.
4.
5.
6.
^
^ M
M M
0
3 M $^
M
re search()
Msearch()
search() None search()

M
M
MM
M
MMM M
MMMM M
search() None
100=C
200=CC
300=CCC
400=CD
500=D
600=DC
700=DCC
800=DCCC
900=CM
CM
CD
0 3 C0 0
D 0 3 C
D 0 3 C
>>>importre
>>>pattern='^M?M?M?(CM|CD|D?C?C?C?)$'
>>>re.search(pattern,'MCM')
<_sre.SRE_Matchobjectat01070390>
>>>re.search(pattern,'MD')
<_sre.SRE_Matchobjectat01073A50>
>>>re.search(pattern,'MMMCCC')
<_sre.SRE_Matchobjectat010748A8>
>>>re.search(pattern,'MCMC')
>>>re.search(pattern,'')
<_sre.SRE_Matchobjectat01071D98>
1.
2.
3.
4.
5.
6.
CMCD D?C?C?C? D 0
3 C
CM
MCM M M
CM CD D?C?C?C?
MCM 1900
MD M M
D?C?C?C? D C
MD 1500
MMMCCC M
D?C?C?C? CCCD MMMCCC
3300
MCMC M M
CM $
CC D?C?C?C? CM
CM
M
D?C?C?C?
{N,M}
{1,4} 1 4
0 3
>>>importre
<_sre.SRE_Matchobjectat0x008EE090>
<_sre.SRE_Matchobjectat0x008EEB48>
>>>
M
M
2.
M
3.
M
4.
M
M None
1.
>>>pattern='^M{0,3}$'
<_sre.SRE_Matchobjectat0x008EEDA8>
>>>
1.
2.
3.
4.
5.
0
3 M 0 3
1 3
M M{1,3}
1 M 0 3
2 M 0 3
3 M 0 3
3 M 0 3
3 M 4
None
>>>pattern='^M?M?M?(CM|CD|D?C?C?C?)(XC|XL|L?X?X?X?)$'
>>>re.search(pattern,'MCMXL')
>>>re.search(pattern,'MCML')
>>>re.search(pattern,'MCMLX')
>>>re.search(pattern,'MCMLXXX')
>>>re.search(pattern,'MCMLXXXX')
>>>
M CM
XLA|B|C AB
C XL XC L?X?X?X?
MCMXL
1940
2.
M CM
L L?X?X?X? L X
MCML 1950
3.
M CM
L X X
MCMLX 1960
1.
M CM
L XMCMLXXX
1980
5.
M CM
L X
X NoneMCMLXXXX
4.
(A|B) A B
>>>pattern=
'^M?M?M?(CM|CD|D?C?C?C?)(XC|XL|L?X?X?X?)(IX|IV|V?I?I?I?)
$'
{n,m}
>>>pattern=
'^M{0,3}(CM|CD|D?C{0,3})(XC|XL|L?X{0,3})(IX|IV|V?I{0,3})
$'
>>>re.search(pattern,'MDLV')
>>>re.search(pattern,'MMDCLXVI')
>>>re.search(pattern,'MMMDCCCLXXXVIII')
>>>re.search(pattern,'I')
^ M{0,3} 0 3
M MD?C{0,3}
D 0 3 C
D C
L?X{0,3} L X V?I{0,3}
V I$MDLV
1555
2.
^ 2 MD?C{0,3}
D 1 CL?X{0,3} L 1
X V?I{0,3} V 1 I
MMDCLXVI 2666
3.
^ 3 MD?C{0,3}
D 3 CL?X{0,3} L 3 X
V?I{0,3} V 3 I
MMMDCCCLXXXVIII 3888
4.
^ M
0 3 D?C{0,3}
D C L?X{0,3}
L XV?I{0,3} V 1 I
1.
python
\
python
#
python
>>>pattern='''
^#beginningofstring
M{0,3}#thousands0to3Ms
(CM|CD|D?C{0,3})#hundreds900(CM),400(CD),
0300(0to3Cs),
#or500800(D,
followedby0to3Cs)
(XC|XL|L?X{0,3})#tens90(XC),40(XL),030
(0to3Xs),
#or5080(L,followed
by0to3Xs)
(IX|IV|V?I{0,3})#ones9(IX),4(IV),03(0
to3Is),
#or58(V,followedby
0to3Is)
$#endofstring
'''
>>>re.search(pattern,'M',re.VERBOSE)
>>>re.search(pattern,'MCMLXXXIX',re.VERBOSE)
>>>re.search(pattern,'MMMDCCCLXXXVIII',re.VERBOSE)
re.VERBOSE
2.
1 M CM L
X IX
3.
3 M D C
X V I
1.
4.
re.VERBOSE
search()
python
:
\d 09 .\D.
8005551212
8005551212
800.555.1212
(800)5551212
18005551212
80055512121234
8005551212x1234
8005551212ext.1234
work1(800)555.1212#1234
800
555 1212
1234
>>>phonePattern=re.compile(r'^(\d{3})(\d{3})
(\d{4})$')
>>>phonePattern.search('8005551212').groups()
('800','555','1212')
>>>phonePattern.search('80055512121234')
AttributeError:'NoneType'objecthasnoattribute
'groups'
(\d{3})\d{3}\d
0 9{3} 3
{n,m} 3
3
4
2.
search()
groups()
1.
4.
search() groups()
search() None
groups()
None.groups()
3.
>>>phonePattern=re.compile(r'^(\d{3})(\d{3})
(\d{4})(\d+)$')
('800','555','1212','1234')
>>>
>>>
2.
groups()
1.
4.
3.
>>>phonePattern=
re.compile(r'^(\d{3})\D+(\d{3})\D+(\d{4})\D+(\d+)$')
('800','555','1212','1234')
('800','555','1212','1234')
>>>
>>>
1.
2.
3.
4.
5.
3
\D+\D
+\D+
\D+
>>>phonePattern=
re.compile(r'^(\d{3})\D*(\d{3})\D*(\d{4})\D*(\d*)$')
('800','555','1212','1234')
>>>phonePattern.search('800.555.1212x1234').groups()
('800','555','1212','1234')
('800','555','1212','')
>>>phonePattern.search('(800)5551212x1234')
>>>
1.
2.
3.
4.
5.
+*
\D+\D*+
3 800
0 3
555 0
4 1212 0
1234
groups() 4
>>>phonePattern=
re.compile(r'^\D*(\d{3})\D*(\d{3})\D*(\d{4})\D*(\d*)$')
>>>phonePattern.search('(800)5551212ext.
1234').groups()
('800','555','1212','1234')
('800','555','1212','')
>>>phonePattern.search('work1(800)555.1212#1234')
>>>
\d* 0
2.
\D*
3.
4 0
0
4.
1
\d*
1.
>>>phonePattern=
re.compile(r'(\d{3})\D*(\d{3})\D*(\d{4})\D*(\d*)$')
>>>phonePattern.search('work1(800)555.1212
#1234').groups()
('800','555','1212','1234')
('800','555','1212','')
('800','555','1212','1234')
2.
3.
4.
1.
>>>phonePattern=re.compile(r'''
#don'tmatchbeginningofstring,
numbercanstartanywhere
(\d{3})#areacodeis3digits(e.g.'800')
\D*#optionalseparatorisanynumberof
nondigits
(\d{3})#trunkis3digits(e.g.'555')
\D*#optionalseparator
(\d{4})#restofnumberis4digits(e.g.
'1212')
\D*#optionalseparator
(\d*)#extensionisoptionalandcanbeany
numberofdigits
$#endofstring
''',re.VERBOSE)
>>>phonePattern.search('work1(800)555.1212
#1234').groups()
('800','555','1212','1234')
('800','555','1212','')
2.
1.
^
$
\b
\d
\D
x? x 0 1 x
x* 0 x
x+ 1 x
x{n,m} n m x n m
(a|b|c) a b c
(x)
re.search groups()
20019MarkPilgrim
:Python3
:

MyspellingisWobbly.ItsgoodspellingbutitWobbles,andthe
lettersgetinthewrongplaces.
WinniethePooh
Borg
SXZESBass
bassesfaxfaxeswaltzwaltzes
HESH
SH
coachcoachesrash
rashesCHSH
cheetahcheetahsH
IYYIESY
Svacancy
vacanciesdaydays
S
Manmenwoman
womenhumanhumansMousemicelouse
licehousehousesKnifekniveswife
wiveslowlifelowlifes
sheepdeerhaiku
Python
[plural1.py]
importre
defplural(noun):
ifre.search('[sxz]$',noun):
returnre.sub('$','es',noun)
elifre.search('[âeioudgkprt]h$',noun):
elifre.search('[âeiou]y$',noun):
returnre.sub('y$','ies',noun)
else:
returnnoun+'s'
[sxz]sxz$
nounsxz
2.
re.sub()
1.
>>>importre
>>>re.search('[abc]','Mark')
<_sre.SRE_Matchobjectat0x001C1FA8>
>>>re.sub('[abc]','o','Mark')
'Mork'
>>>re.sub('[abc]','o','rock')
'rook'
>>>re.sub('[abc]','o','caps')
'oops'
Markabca
abcoMark
Mork
3.
rockrook
4.
capsoaps
re.sub
capsoopsca
o
1.
2.
plural()
defplural(noun):
ifre.search('[sxz]$',noun):
elifre.search('[âeioudgkprt]h$',noun):
elifre.search('[âeiou]y$',noun):
else:
returnnoun+'s'
$es
es
noun+'es'
2.
^[âbc]abc
[âeioudgkprt]ae
ioudgkprt
hH
H
3.
YY
aeiouYY
I
1.
>>>importre
>>>re.search('[âeiou]y$','vacancy')
<_sre.SRE_Matchobjectat0x001C1FA8>
>>>re.search('[âeiou]y$','boy')
>>>
>>>re.search('[âeiou]y$','day')
>>>
>>>re.search('[âeiou]y$','pita')
>>>
1.
vacancycyc
aeiou
boyoyy
2.
odayay
3.
pitay
>>>re.sub('y$','ies','vacancy')
'vacancies'
>>>re.sub('y$','ies','agency')
'agencies'
>>>re.sub('([âeiou])y$',r'\1ies','vacancy')
'vacancies'
vacancyvacanciesagency
agenciesboy
boies
re.searchre.sub
2.
y
\1
yc
cciesy
\2\3
1.
\1
SXZES
SXZES
[downloadplural2.py]
importre
defmatch_sxz(noun):
returnre.search('[sxz]$',noun)
defapply_sxz(noun):
defmatch_h(noun):
returnre.search('[âeioudgkprt]h$',noun)
defapply_h(noun):
defmatch_y(noun):
returnre.search('[âeiou]y$',noun)
defapply_y(noun):
defmatch_default(noun):
returnTrue
defapply_default(noun):
returnnoun+'s'
rules=((match_sxz,apply_sxz),
(match_h,apply_h),
(match_y,apply_y),
(match_default,apply_default)
)
defplural(noun):
formatches_rule,apply_ruleinrules:
ifmatches_rule(noun):
returnapply_rule(noun)
re.search()
2.
re.sub()
3.
rules
plural()
4.
plural()
forrules
1.

for
matches_rulematch_sxzapply_rule
apply_sxz
matches_rulematch_happly_rule
apply_h
(match_default)True
(apply_default)
rules
Python
rules
formatches_rule
apply_rulefor
matches_sxz(noun)
apply_sxz(noun)
for
defplural(noun):
ifmatch_sxz(noun):
returnapply_sxz(noun)
ifmatch_h(noun):
returnapply_h(noun)
ifmatch_y(noun):
returnapply_y(noun)
ifmatch_default(noun):
returnapply_default(noun)
plural()
1.
2.
3.
plural()
ifplural()
match_foo()apply_foo()rules
rules
re.search()
re.sub()
importre
defbuild_match_and_apply_functions(pattern,search,
replace):
defmatches_rule(word):
returnre.search(pattern,word)
defapply_rule(word):
returnre.sub(search,replace,word)
return(matches_rule,apply_rule)
1.
build_match_and_apply_functions()
patternsearchreplace
matches_rule()
build_match_and_apply_functions()pattern
matchs_rules()wordre.search()
2.
searchreplaceapply_rule()
wordre.sub()
closures
word
searchreplace
3.
match_rule() patternapply_rule()
searchreplace
patterns=\
(
('[sxz]$','$','es'),
('[âeioudgkprt]h$','$','es'),
('(qu|[âeiou])y$','y$','ies'),
('$','$','s')
)
rules = [build_match_and_apply_functions(pattern, search,
replace)
for(pattern,search,replace)inpatterns]
re.search()
re.sub()
2.
match_default()
True
s
$
match_default()True
1.

s
3.
patterns
re.search()re.sub()
plural()
defplural(noun):
formatches_rule,apply_ruleinrules:
1.
plural()
plural()
plural4rules.txt.
[downloadplural4rules.txt]
[sxz]$$es
[âeioudgkprt]h$$es
[âeiou]y$y$ies
$$s
importre
defbuild_match_and_apply_functions(pattern,search,
replace):
defmatches_rule(word):
returnre.search(pattern,word)
defapply_rule(word):
returnre.sub(search,replace,word)
return(matches_rule,apply_rule)
rules=[]
withopen('plural4rules.txt',encoding='utf8')as
pattern_file:
forlineinpattern_file:
pattern,search,replace=line.split(None,3)
rules.append(build_match_and_apply_functions(
pattern,search,replace))
1.
2.
open()
with
contextwith
Pythonwith
with
forlinein<fileobject>
3.
line
4.
split()split()None
[sxz]$$es['[sxz]$','$',
'es']pattern'[sxz]$'search'$'
replace'es'
5.
patternsearchreplace
rulesrules
plural()
plural()
plural()plural()
defrules(rules_filename):
withopen(rules_filename,encoding='utf8')as
pattern_file:
pattern,search,replace=line.split(None,
3)
yield
build_match_and_apply_functions(pattern, search, replace)
defplural(noun,rules_filename='plural5rules.txt'):
formatches_rule,apply_rulein
rules(rules_filename):
raiseValueError('nomatchingrulefor
{0}'.format(noun))
>>>defmake_counter(x):
...print('enteringmake_counter')
...whileTrue:
...yieldx
...print('incrementingx')
...x=x+1
...
>>>counter=make_counter(2)
>>>counter
<generatorobjectat0x001C9C10>
>>>next(counter)
enteringmake_counter
2
>>>next(counter)
incrementingx
3
>>>next(counter)
incrementingx
4
1.
2.
3.
4.
5.
6.
make_counteryield
x
Generator
make_counter
make_counter()print()
make_counter()
next()
counternext()yield
make_counter()
2make_counter(2)
next()
yield
yieldnext()
print()incrementingx
x=x+1while
yieldx
x3
next(counter)
x4
make_counter
x

yieldnext()
[downloadfibonacci.py]
deffib(max):
a,b=0,1
whilea<max:
yielda
a,b=b,a+b
01
0a1b
ayield
2.
3.
ba
(a+b)b
a3b5a,b=b,a+ba
5bb8ab
1.
for
>>>fromfibonacciimportfib
>>>forninfib(1000):
...print(n,end='')
01123581321345589144233377610987
>>>list(fib(1000))
[0,1,1,2,3,5,8,13,21,34,55,89,144,233,377,
610,987]
forfib()for
next()fib()
forn
2.
fornfib()yield
fib()a
max1000for
3.
list()
for
1.
plural5.pyplural()
defrules(rules_filename):
withopen(rules_filename,encoding='utf8')as
pattern_file:
pattern,search,replace=line.split(None,
3)
yield
build_match_and_apply_functions(pattern, search, replace)

defplural(noun,rules_filename='plural5rules.txt'):
formatches_rule,apply_rulein
rules(rules_filename):
raiseValueError('nomatchingrulefor
{0}'.format(noun))
line.split(None,3)
2.
yield
rules()
3.
rules()forfor
rules()
for
rules()forlinein
pattern_file
1.
plural4
plural()
plural()rules()
import
Python
PEP255:
Pythonwith
Python
20019MarkPilgrim
:Python3
:
&
yield
[fibonacci2.py]
classFib:
''''''
def__init__(self,max):
self.max=max
def__iter__(self):
self.a=0
self.b=1
returnself
def__next__(self):
fib=self.a
iffib>self.max:
raiseStopIteration
self.a,self.b=self.b,self.a+self.b
returnfib
classFib:
(class)
Python
Python
Python
class
classPapayaWhip:
pass
PapayaWhip
EachWordLikeThis
2.
iffor
1.
PapayaWhip
pass Python
Python pass JavaC

({})
Python
C++Python
Python
__init__()
__INIT__()
Fib__init__
classFib:
''''''
() docstring
2.
__init__()
C++__init__()
__init__()
1.
__init__()
self C++ Java
thisselfPython
self
__init__()self
selfPython
Python
__init__()
>>>importfibonacci2
>>>fib=fibonacci2.Fib(100)
>>>fib
<fibonacci2.Fibobjectat0x00DB8810>
>>>fib.__class__
<class'fibonacci2.Fib'>
>>>fib.__doc__
''
Fib fibonacci2
fib100 Fib
__init__() max
fibFib
2.
3.
__class__
JavaClassgetName()
getSuperclass()Python
4.
docstring
docstring
1.
Python
C++Java
new
classFib:
self.max=max
1.
self.max
__init__()max self.max
classFib:
self.max=max
.
.
.
def__next__(self):
fib=self.a
iffib>self.max:
1.
2.
self.max__init__()
__next__()
Fib
>>>importfibonacci2
>>>fib1=fibonacci2.Fib(100)
>>>fib2=fibonacci2.Fib(200)
>>>fib1.max
100
>>>fib2.max
200
__iter__()
__init____iter____next__
_
Python
[fibonacci2.py]
classFib:
self.max=max
def__iter__(self):
self.a=0
self.b=1
returnself
def__next__(self):
fib=self.a
iffib>self.max:
raiseStopIteration
self.a,self.b=self.b,self.a+self.b
returnfib
1.
2.
3.
4.
5.
6.
fib
Fib(max) max
__init__()__init__()
iter(fib)__iter__()
for
self.aself.b__iter__()
__next__()
__iter__()self
__next__()
next()__next__()
__next__()StopIteration
forStopIteration
for
__next__()
return yield
return
>>>fromfibonacci2importFib
>>>forninFib(1000):
...print(n,end='')
01123581321345589144233377610987
Fibonacci
asagenerator
for
forFib(1000) Fib
fib_inst
foriter(fib_inst)
fib_iterfib_iter==
fib_inst__iter__()self for
fornext(fib_iter)
fib_iter __next__()
forn n for
for
next(fib_iter)StopIterationfor
StopIteration__next__()
iter(f)f.__iter__
next(f)f.__next__
[plural6.py]
classLazyRules:
rules_filename='plural6rules.txt'
def__init__(self):
self.pattern_file=open(self.rules_filename,
encoding='utf8')
self.cache=[]
def__iter__(self):
self.cache_index=0
returnself
def__next__(self):
self.cache_index+=1
iflen(self.cache)>=self.cache_index:
returnself.cache[self.cache_index1]
ifself.pattern_file.closed:
raiseStopIteration
line=self.pattern_file.readline()
ifnotline:
self.pattern_file.close()
raiseStopIteration
funcs=build_match_and_apply_functions(
pattern,search,replace)
self.cache.append(funcs)
returnfuncs
rules=LazyRules()
__iter__()__next__()
rules
import
classLazyRules:
rules_filename='plural6rules.txt'
def__init__(self):
self.pattern_file=open(self.rules_filename,
encoding='utf8')
self.cache=[]
LazyRules
2.
__next__()
1.
rules_filename__iter__()
self.rules_filenameLazyRules
>>>importplural6
>>>r1=plural6.LazyRules()
>>>r2=plural6.LazyRules()
>>>r1.rules_filename
'plural6rules.txt'
'plural6rules.txt'
>>>r2.rules_filename='r2override.txt'
'r2override.txt'
'plural6rules.txt'
>>>r2.__class__.rules_filename
'plural6rules.txt'
>>>r2.__class__.rules_filename='papayawhip.txt'
'papayawhip.txt'
'r2overridetxt'
1.
2.
3.
4.
5.
rules_filename
__class__
r1
overriddenr2
def__iter__(self):
self.cache_index=0
returnself
foriter(rules)
__iter__()
2.
__iter__()
self__next__()
1.
def__next__(self):
.
.
.
funcs=
build_match_and_apply_functions(
pattern,search,replace)
self.cache.append(funcs)
returnfuncs
for__next__()
next(rules)
2.
3.
funcs self.cache
1.
def__next__(self):
.
.
.
line=self.pattern_file.readline()
ifnotline:
self.pattern_file.close()
raiseStopIteration
.
.
.
readline()
readlines()
2.
readline()line
line'\n'
line
3.
StopIteration
1.
__next__()
def__next__(self):
self.cache_index+=1
iflen(self.cache)>=self.cache_index:
returnself.cache[self.cache_index1]
ifself.pattern_file.closed:
raiseStopIteration
.
.
.
1.
self.cache
self.cache_index
self.cacheself.cache_index
2.
LazyRules
rules
plural()
plural() for
iter(rules)
for rules
__next__()
forrules
__next__()
__next__()
readline()
import
2.
3.
1.
LazyRules
__init__()
Python LazyRules
Python
Python
Python
LazyRules
__init__()
tell()

seek()
PEP234:Iterators
PEP255:SimpleGenerators
GeneratorTricksforSystems
Programmers
20019MarkPilgrim
Search
:Python3
:
Greatfleashavelittlefleasupontheirbackstobiteem,
Andlittlefleashavelesserfleas,andsoadinfinitum.
AugustusDeMorgan
HAWAII+IDAHO+IOWA+OHIO==STATES.,
510199+98153+9301+3593==621246.
HAWAII+IDAHO+IOWA+OHIO==STATES
510199+98153+9301+3593==621246
H=5
A=1
W=0
I=9
D=8
O=3
S=6
T=2
E=4
cryptarithms(alphametics)
09
,#8220;
,
0 SEND+MORE=
MONEY
RaymondHettinger
Python 14
[alphametics.py]
importre
importitertools
defsolve(puzzle):
words=re.findall('[AZ]+',puzzle.upper())
unique_characters=set(''.join(words))
assertlen(unique_characters)<=10,'Toomany
letters'
first_letters={word[0]forwordinwords}
n=len(first_letters)
sorted_characters=''.join(first_letters)+\
''.join(unique_charactersfirst_letters)
characters=tuple(ord(c)forcinsorted_characters)
digits=tuple(ord(c)forcin'0123456789')
zero=digits[0]
forguessinitertools.permutations(digits,
len(characters)):
ifzeronotinguess[:n]:
equation=
puzzle.translate(dict(zip(characters,guess)))
ifeval(equation):
returnequation
importsys
forpuzzleinsys.argv[1:]:
print(puzzle)
solution=solve(puzzle)
ifsolution:
print(solution)
Linux ,
(
)
alphametics.py"HAWAII+IDAHO+IOWA+OHIO==STATES"
HAWAII+IDAHO+IOWA+OHIO=STATES
510199+98153+9301+3593==621246
alphametics.py"I+LOVE+YOU==DORA"
I+LOVE+YOU==DORA
1+2784+975==3760
alphametics.py"SEND+MORE==MONEY"
SEND+MORE==MONEY
9567+1085==10652
(A
Z)
>>>importre
>>>re.findall('[09]+','162by4sinrowsof8')
['16','2','4','8']
>>>re.findall('[AZ]+','SEND+MORE==MONEY')
['SEND','MORE','MONEY']
1.
rePython
findall()
findall()
2.
>>>re.findall('s.*?s',"Thesixthsicksheikh'ssixth
sheep'ssick.")
['sixths',"sheikh'ss","sheep'ss"]
s,
(.*?),, s
:
1.
2.
3.
4.
5.
Thesixthsicksheikh'ssixthsheep'ssick.
re.findall() 3
Sets
>>>a_list=['The','sixth','sick',"sheik's",'sixth',
"sheep's",'sick']
>>>set(a_list)
{'sixth','The',"sheep's",'sick',"sheik's"}
>>>a_string='EASTISEAST'
>>>set(a_string)
{'A','','E','I','S','T'}
>>>words=['SEND','MORE','MONEY']
>>>''.join(words)
'SENDMOREMONEY'
>>>set(''.join(words))
{'E','D','M','O','N','S','R','Y'}
set()
for
, Python
2.
3.
,''.join(a_list)
4.
1.
unique_characters=set(''.join(words))
Python assert
>>>assert1+1==2
>>>assert1+1==3
AssertionError
>>>assert2+2==5,"Onlyforverylargevaluesof2"
AssertionError:Onlyforverylargevaluesof2
1.
assert Python
,1+1==2True,assert
2.
, PythonFalse,assert
AssertionError.
3.
AssertionError
,:
assertlen(unique_characters)<=10,'Toomanyletters'
:
iflen(unique_characters)>10:
raiseAssertionError('Toomanyletters')
assert 10
10 , 10
>>>unique_characters={'E','D','M','O','N','S',
'R','Y'}
>>>gen=(ord(c)forcinunique_characters)
>>>gen
<generatorobject<genexpr>at0x00BADC10>
>>>next(gen)
69
>>>next(gen)
68
>>>tuple(ord(c)forcinunique_characters)
(69,68,77,79,78,83,82,89)
yield
,
2.
3.
next(gen)
4.
tuple(),list(),
set()
ord(c)
forcinunique_characterstuple(),
Python
1.
CPU(RAM)
( tuple()set()),
!
,:
deford_map(a_string):
forcina_string:
yieldord(c)
gen=ord_map(unique_characters)
!
,?(
,
,)
(
)
1
3 2 , 3
>>>importitertools
>>>perms=itertools.permutations([1,2,3],2)
>>>next(perms)
(1,2)
>>>next(perms)
(1,3)
>>>next(perms)
(2,1)
>>>next(perms)
(2,3)
>>>next(perms)
(3,1)
>>>next(perms)
(3,2)
>>>next(perms)
StopIteration
1.
itertools
permutations()
2.
permutations()( 3
)
for
3.
[1,2,3] 2 (1,2)
4.
:(2,1)(1,2)
5.
[1,2,3](1,1)
(2,2)
StopIteration
itertools
permutations()
>>>importitertools
>>>perms=itertools.permutations('ABC',3)
>>>next(perms)
('A','B','C')
>>>next(perms)
('A','C','B')
>>>next(perms)
('B','A','C')
>>>next(perms)
('B','C','A')
>>>next(perms)
('C','A','B')
>>>next(perms)
('C','B','A')
>>>next(perms)
StopIteration
>>>list(itertools.permutations('ABC',3))
[('A','B','C'),('A','C','B'),
('B','A','C'),('B','C','A'),
('C','A','B'),('C','B','A')]
'ABC'
['A','B','C']
['A','B','C'] 3 ('A','B','C')
2.
5 3
3.
permutations()
list()
1.
ITERTOOLS
>>>importitertools
>>>list(itertools.product('ABC','123'))
[('A','1'),('A','2'),('A','3'),
('B','1'),('B','2'),('B','3'),
('C','1'),('C','2'),('C','3')]
>>>list(itertools.combinations('ABC',2))
[('A','B'),('A','C'),('B','C')]
1.
itertools.product()
2.
itertools.combinations()
itertools.permutations()
itertools.permutations('ABC',2)('A','B')and
('B','A')(),
itertools.combinations('ABC',2)('B','A')
('A','B')
[favoritepeople.txt]
>>>names=list(open('examples/favoritepeople.txt',
encoding='utf8'))
>>>names
['Dora\n','Ethan\n','Wesley\n','John\n','Anne\n',
'Mike\n','Chris\n','Sarah\n','Alex\n','Lizzie\n']
>>>names=[name.rstrip()fornameinnames]
>>>names
['Dora','Ethan','Wesley','John','Anne',
'Mike','Chris','Sarah','Alex','Lizzie']
>>>names=sorted(names)
>>>names
['Alex','Anne','Chris','Dora','Ethan',
'John','Lizzie','Mike','Sarah','Wesley']
>>>names=sorted(names,key=len)
>>>names
['Alex','Anne','Dora','John','Mike',
'Chris','Ethan','Sarah','Lizzie','Wesley']
1.
(),list(open(filename))
rstrip()(
lstrip() strip())
sorted()
3.
4.
sorted() key,
key len(),
len(eachitem)
2.
itertools ?
continuingfromthepreviousinteractiveshell
>>>importitertools
>>>groups=itertools.groupby(names,len)
>>>groups
<itertools.groupbyobjectat0x00BB20C0>
>>>list(groups)
[(4,<itertools._grouperobjectat0x00BA8BF0>),
(5,<itertools._grouperobjectat0x00BB4050>),
(6,<itertools._grouperobjectat0x00BB4030>)]
>>>groups=itertools.groupby(names,len)
>>>forname_length,name_iteringroups:
...print('Nameswith{0:d}
letters:'.format(name_length))
...fornameinname_iter:
...print(name)
...
Nameswith4letters:
Alex
Anne
Dora
John
Mike
Nameswith5letters:
Chris
Ethan
Sarah
Nameswith6letters:
Lizzie
Wesley
1.
itertools.groupby() key,
key_function(eachitem)
key
2.
list(),
(
, for ), itertools.groupby()
3.
,
itertools.groupby(names,len) 4
5
groupby();
, key
itertools.groupby()
len()
Areyouwatchingclosely?
>>>list(range(0,3))
[0,1,2]
>>>list(range(10,13))
[10,11,12]
>>>list(itertools.chain(range(0,3),range(10,13)))
[0,1,2,10,11,12]
>>>list(zip(range(0,3),range(10,13)))
[(0,10),(1,11),(2,12)]
>>>list(zip(range(0,3),range(10,14)))
[(0,10),(1,11),(2,12)]
>>>list(itertools.zip_longest(range(0,3),range(10,
14)))
[(0,10),(1,11),(2,12),(None,13)]
1.
itertools.chain()
(
)
2.
zip():
zip()range(10,14)
3.
(10,11,12,13),range(0,3) 3 ,zip()
3
4.
itertools.zip_longest()
, None .
?:
>>>characters=('S','M','E','D','O','N','R','Y')
>>>guess=('1','2','0','3','4','5','6','7')
>>>tuple(zip(characters,guess))
(('S','1'),('M','2'),('E','0'),('D','3'),
('O','4'),('N','5'),('R','6'),('Y','7'))
>>>dict(zip(characters,guess))
{'E':'0','D':'3','M':'2','O':'4',
'N':'5','S':'1','R':'6','Y':'7'}
( 1
),zip
2.
? dict()
(
)
(
#8220;),characters
guess
1.
characters=tuple(ord(c)forcinsorted_characters)
digits=tuple(ord(c)forcin'0123456789')
...
forguessinitertools.permutations(digits,
len(characters)):
...
equation=puzzle.translate(dict(zip(characters,
guess)))
translate()?,
Python
:lower(),count(),format()
:translate()
>>>translation_table={ord('A'):ord('O')}
>>>translation_table
{65:79}
>>>'MARK'.translate(translation_table)
'MORK'
byte)
2.
Python3ord()
ASCII AZ, 6590
3.
translate()
MARKMORK.
1.
>>>characters=tuple(ord(c)forcin'SMEDONRY')
>>>characters
(83,77,69,68,79,78,82,89)
>>>guess=tuple(ord(c)forcin'91570682')
>>>guess
(57,49,53,55,48,54,56,50)
>>>translation_table=dict(zip(characters,guess))
>>>translation_table
{68:55,69:53,77:49,78:54,79:48,82:56,83:57,
89:50}
>>>'SEND+MORE==MONEY'.translate(translation_table)
'9567+1085==10652'
,
charactersalphametics.solve()
sorted_characters.
2.
guess,alphametics.solve()
3.
charactersguesszipping
alphametics.solve()for
4.
translate()
( characters
guess )
Python
1.
Python
PYTHON
(,)
'9567+1085==
10652'
eval(),Python
>>>eval('1+1==2')
True
>>>eval('1+1==3')
False
>>>eval('9567+1085==10652')
True
!eval()
Python
>>>eval('"A"+"B"')
'AB'
>>>eval('"MARK".translate({65:79})')
'MORK'
>>>eval('"AAAAA".count("A")')
5
>>>eval('["*"]*5')
['*','*','*','*','*']
!
>>>x=5
>>>eval("x*5")
25
>>>eval("pow(x,2)")
25
>>>importmath
>>>eval("math.sqrt(x)")
2.2360679774997898
1.
2.
3.
eval() eval()
(eval()),
>>>importsubprocess
>>>eval("subprocess.getoutput('ls~')")
'DesktopLibraryPictures\
DocumentsMoviesPublic\
MusicSites'
>>>eval("subprocess.getoutput('rm/some/random/file')")
1.
subprocess shell
2.
shell
__import__()
eval()
:
>>>eval("__import__('subprocess').getoutput('rm
/some/random/file')")
1.
'rmrf~'
,
eval()
:
webGosh,
(),
eval()
>>>x=5
>>>eval("x*5",{},{})
File"<string>",line1,in<module>
NameError:name'x'isnotdefined
>>>eval("x*5",{"x":x},{})
>>>importmath
>>>eval("math.sqrt(x)",{"x":x},{})
NameError:name'math'isnotdefined
eval()
"x*5",
x,eval()
2.
3.
math , eval()
1.
Web
>>>eval("pow(5,2)",{},{})
25
>>>eval("__import__('math').sqrt(5)",{},{})
2.2360679774997898
Python pow(5,2)
,52 pow()
2.
(),__import__()
1.
eval()
>>>eval("__import__('subprocess').getoutput('rm
/some/random/file')",{},{})
. web
eval()?,
>>>eval("__import__('math').sqrt(5)",
...{"__builtins__":None},{})
NameError:name'__import__'isnotdefined
>>>eval("__import__('subprocess').getoutput('rmrf
/')",
NameError:name'__import__'isnotdefined
,
"__builtins__"None(Python)
.,"__builtins__"
()
__builtins____builtin__,
__builtins__,
eval()?
>>>eval("2**2147483647",
1.
__builtins__,
, 22147483647
CPU 100%( shell
,CtrlC )
,
,Python,
1.
2.
3.
4.
5.
6.
7.
8.
re.findall()
set()
assert 10 (
)
ASCII
translate()Python
eval()Python
True
14 .
itertools
itertools
RaymondHettingerPyCon2009EasyAIwith
Python
Recipe576615:Alphameticssolver,RaymondHettinger
Python2
MoreofRaymondHettingersrecipesintheActiveStateCode
repository

,
RaymondHettinger
Python3
20019MarkPilgrim
:Python3
:
Certitudeisnotthetestofcertainty.Wehavebeencocksureof
manythingsthatwerenotso.
OliverWendellHolmes,Jr.
1.
2.
3.
4.
5.
6.
1
3999
1000
13999
0
roman.py
to_roman()from_roman()to_roman()1
3999
to_roman
TDD
to_roman()from_roman()
Python
unittest
...
to_roman() 13999
1.
__init__
__main__
[downloadromantest1.py]
importroman1
importunittest
classKnownValues(unittest.TestCase):
known_values=((1,'I'),
(2,'II'),
(3,'III'),
(4,'IV'),
(5,'V'),
(6,'VI'),
(7,'VII'),
(8,'VIII'),
(9,'IX'),
(10,'X'),
(50,'L'),
(100,'C'),
(500,'D'),
(1000,'M'),
(31,'XXXI'),
(148,'CXLVIII'),
(294,'CCXCIV'),
(312,'CCCXII'),
(421,'CDXXI'),
(528,'DXXVIII'),
(621,'DCXXI'),
(782,'DCCLXXXII'),
(870,'DCCCLXX'),
(941,'CMXLI'),
(1043,'MXLIII'),
(1110,'MCX'),
(1226,'MCCXXVI'),
(1301,'MCCCI'),
(1485,'MCDLXXXV'),
(1509,'MDIX'),
(1607,'MDCVII'),
(1754,'MDCCLIV'),
(1832,'MDCCCXXXII'),
(1993,'MCMXCIII'),
(2074,'MMLXXIV'),
(2152,'MMCLII'),
(2212,'MMCCXII'),
(2343,'MMCCCXLIII'),
(2499,'MMCDXCIX'),
(2574,'MMDLXXIV'),
(2646,'MMDCXLVI'),
(2723,'MMDCCXXIII'),
(2892,'MMDCCCXCII'),
(2975,'MMCMLXXV'),
(3051,'MMMLI'),
(3185,'MMMCLXXXV'),
(3250,'MMMCCL'),
(3313,'MMMCCCXIII'),
(3408,'MMMCDVIII'),
(3501,'MMMDI'),
(3610,'MMMDCX'),
(3743,'MMMDCCXLIII'),
(3844,'MMMDCCCXLIV'),
(3888,'MMMDCCCLXXXVIII'),
(3940,'MMMCMXL'),
(3999,'MMMCMXCIX'))
deftest_to_roman_known_values(self):
'''to_romanshouldgiveknownresultwithknown
input'''
forinteger,numeralinself.known_values:
result=roman1.to_roman(integer)
self.assertEqual(numeral,result)
unittest.main()
unittest
TestCaseTestCase
2.
3.
4.
to_roman().
;
to_roman()
to_roman()
1.
unittest to_roman()
to_roman()
5.
to_roman()
rightTestCase
assertEqualto_roman()
(result) g(numeral)
assertEqual
to_roman()
assertEqual
test_to_roman_known_values
to_roman()
to_roman()
#roman1.py
defto_roman(n):
'''convertintegertoRomannumeral'''
pass
1.
to_roman()API
Python pass
romantest1.pyv
romantest1.pyv
test_to_roman_known_values(__main__.KnownValues)
to_romanshouldgiveknownresultwithknowninput...
FAIL
========================================================
==============
FAIL:to_romanshouldgiveknownresultwithknowninput
File"romantest1.py",line73,in
self.assertEqual(numeral,result)
AssertionError:'I'!=None
Ran1testin0.016s
FAILED(failures=1)
1.
2.
3.
4.
5.
unittest.main(),
romantest.py
;
unittest.TestCase
unittest
docstring
unittest
assertEqual()
AssertionErrorto_roman(1)'I'
PythonNone
unittest
unittest
assertXYZassertRaises
assertEqual
to_roman()
[downloadroman1.py]
roman_numeral_map=(('M',1000),
('CM',900),
('D',500),
('CD',400),
('C',100),
('XC',90),
('L',50),
('XL',40),
('X',10),
('IX',9),
('V',5),
('IV',4),
('I',1))
defto_roman(n):
result=''
fornumeral,integerinroman_numeral_map:
whilen>=integer:
result+=numeral
n=integer
returnresult
1.
roman_numeral_map
MI
()
CM
to_roman()
2.
roman_numeral_map
roman_numeral_map
result
to_roman()while
print()
whilen>=integer:
result+=numeral
n=integer
print('subtracting{0}frominput,adding{1}to
output'.format(integer,numeral))
print()
>>>importroman1
>>>roman1.to_roman(1424)
subtracting1000frominput,addingMtooutput
subtracting400frominput,addingCDtooutput
subtracting10frominput,addingXtooutput
subtracting10frominput,addingXtooutput
subtracting4frominput,addingIVtooutput
'MCDXXIV'
to_roman()
romantest1.pyv
ok
Ran1testin0.016s
OK
1.
to_roman()knownvalues
39993888
Python
>>>importroman1
'MMMM'
'MMMMM'
'MMMMMMMMM'
1.
Python
3999to_roman()
OutOfRangeError
classToRomanBadInput(unittest.TestCase):
deftest_too_large(self):
'''to_romanshouldfailwithlargeinput'''
self.assertRaises(roman2.OutOfRangeError,
roman2.to_roman,4000)
unittest.TestCase
2.
test
unittest.TestCase eassertRaises
3.
assertRaisesassertRaises
1.
to_roman()
try...exceptassertRaises
assertRaises
roman2.OutOfRangeErrorto_roman()
4000assertRaises
to_roman()roman2.OutOfRangeError
to_roman();
Python
romantest2.pyv
ok
test_too_large(__main__.ToRomanBadInput)
to_romanshouldfailwithlargeinput...ERROR
========================================================
==============
ERROR:to_romanshouldfailwithlargeinput
File"romantest2.py",line78,intest_too_large
AttributeError:'module'objecthasnoattribute
'OutOfRangeError'
Ran2testsin0.000s
FAILED(errors=1)
2.
OutOfRangeError
assertRaises()
assertRaises()
to_roman()
1.
roman2.py
OutOfRangeError
classOutOfRangeError(ValueError):
pass
ValueError
Exception
2.
pass
Python
1.
romantest2.pyv
ok
to_romanshouldfailwithlargeinput...FAIL
========================================================
==============
FAIL:to_romanshouldfailwithlargeinput
File"romantest2.py",line78,intest_too_large
AssertionError:OutOfRangeErrornotraisedbyto_roman
Ran2testsin0.016s
FAILED(failures=1)
assertRaises()
to_roman()
2.
to_roman()OutOfRangeError
1.
[downloadroman2.py]
defto_roman(n):
ifn>3999:
raiseOutOfRangeError('numberoutofrange(must
belessthan4000)')
result=''
whilen>=integer:
result+=numeral
n=integer
returnresult
1.
(n) 3999
OutOfRangeError
romantest2.pyv
ok
to_romanshouldfailwithlargeinput...ok
Ran2testsin0.000s
OK
1.
MORE HALTING, MORE FIRE
0
>>>importroman2
''
''
deftest_zero(self):
'''to_romanshouldfailwith0input'''
roman3.to_roman,0)
deftest_negative(self):
'''to_romanshouldfailwithnegativeinput'''
roman3.to_roman,1)
1.
test_too_large()
2.
test_zero()test_too_large()
nunittest.TestCase
assertRaises()0to_roman()
OutOfRangeError
test_negative()to_roman()
3.
1
OutOfRangeError
romantest3.pyv
ok
test_negative(__main__.ToRomanBadInput)
to_romanshouldfailwithnegativeinput...FAIL
test_zero(__main__.ToRomanBadInput)
to_romanshouldfailwith0input...FAIL

========================================================
==============
FAIL:to_romanshouldfailwithnegativeinput
File"romantest3.py",line86,intest_negative
roman3.to_roman,1)
========================================================
==============
FAIL:to_romanshouldfailwith0input
File"romantest3.py",line82,intest_zero
roman3.to_roman,0)
Ran4testsin0.000s
FAILED(failures=2)
[downloadroman3.py]
defto_roman(n):
ifnot(0<n<4000):
be1..3999)')
result=''
whilen>=integer:
result+=numeral
n=integer
returnresult
1.
Python
ifnot((0<n)and(n<4000))
0
2.
unittest
romantest3.pyv
ok
to_romanshouldfailwithnegativeinput...ok
to_romanshouldfailwith0input...ok
Ran4testsin0.016s
OK
>>>importroman3
>>>roman3.to_roman(0.5)
''
>>>roman3.to_roman(1.0)
'I'
1.
2.
NotIntegerError
#roman4.py
classOutOfRangeError(ValueError):pass
classNotIntegerError(ValueError):pass
NotIntegerError
.
.
.
deftest_non_integer(self):
'''to_romanshouldfailwithnoninteger
input'''
self.assertRaises(roman4.NotIntegerError,
roman4.to_roman,0.5)
romantest4.pyv
ok
test_non_integer(__main__.ToRomanBadInput)
to_romanshouldfailwithnonintegerinput...FAIL
========================================================
==============
FAIL:to_romanshouldfailwithnonintegerinput
File"romantest4.py",line90,intest_non_integer
self.assertRaises(roman4.NotIntegerError,
roman4.to_roman,0.5)
AssertionError:NotIntegerErrornotraisedbyto_roman
Ran5testsin0.000s
FAILED(failures=1)
defto_roman(n):
ifnot(0<n<4000):
be1..3999)')
ifnotisinstance(n,int):
raiseNotIntegerError('nonintegerscannotbe
converted')

result=''
whilen>=integer:
result+=numeral
n=integer
returnresult
isinstance()
2.
nintNotIntegerError
1.
romantest4.pyv
ok
test_non_integer(__main__.ToRomanBadInput)
to_romanshouldfailwithnonintegerinput...ok
Ran5testsin0.000s
OK
to_roman()
from_roman()
from_roman()to_roman()
deftest_from_roman_known_values(self):
'''from_romanshouldgiveknownresultwith
knowninput'''
forinteger,numeralinself.known_values:
result=roman5.from_roman(numeral)
self.assertEqual(integer,result)
to_roman()
;from_roman()
to_roman()
n=from_roman(to_roman(n))forallvaluesofn
1 3999
to_roman()
1 3999 1 3999
classRoundtripCheck(unittest.TestCase):
deftest_roundtrip(self):
'''from_roman(to_roman(n))==nforalln'''
forintegerinrange(1,4000):
numeral=roman5.to_roman(integer)
from_roman()
romantest5.py
E.E....
========================================================
==============
ERROR:test_from_roman_known_values
(__main__.KnownValues)
from_romanshouldgiveknownresultwithknowninput
test_from_roman_known_values
'from_roman'
========================================================
==============
ERROR:test_roundtrip(__main__.RoundtripCheck)
from_roman(to_roman(n))==nforalln
File"romantest5.py",line103,intest_roundtrip
'from_roman'
Ran7testsin0.019s
FAILED(errors=2)
#roman5.py
deffrom_roman(s):
'''convertRomannumeraltointeger'''
docstring
Python
romantest5.py
F.F....
========================================================
==============
FAIL: test_from_roman_known_values (__main__.KnownValues)
from_romanshouldgiveknownresultwithknowninput
AssertionError:1!=None
========================================================
==============
FAIL:test_roundtrip(__main__.RoundtripCheck)
from_roman(to_roman(n))==nforalln
File"romantest5.py",line104,intest_roundtrip
AssertionError:1!=None
Ran7testsin0.002s
FAILED(failures=2)
from_roman()
deffrom_roman(s):
"""convertRomannumeraltointeger"""
result=0
index=0
whiles[index:index+len(numeral)]==numeral:
result+=integer
index+=len(numeral)
returnresult
1.
to_roman()
()
from_roman()while
print
deffrom_roman(s):
"""convertRomannumeraltointeger"""
result=0
index=0
result+=integer
index+=len(numeral)
print('found',numeral,'oflength',
len(numeral),',adding',integer)
>>>importroman5
>>>roman5.from_roman('MCMLXXII')
foundMoflength1,adding1000
foundCMoflength2,adding900
foundLoflength1,adding50
foundXoflength1,adding10
foundXoflength1,adding10
foundIoflength1,adding1
foundIoflength1,adding1
1972
romantest5.py
.......
Ran7testsin0.060s
OK
from_roman()
(
to_roman()
andfrom_roman()to_roman()
from_roman()
(
)
s
MDCLXVI
I1,II2, III3.VI6(
,51),VII7,VIII8
(IXCM)4
5
4 IIII IV(51)40
XL(5010)41XLI42XLII43
XLIII44XLIV(501051)
9
8VIII9IX
101) VIIIII
90XC900CM
510XVV
100CLL
DC600;
CD(400500100)CI101;IC
1001
XCIX100 10101
from_roman()
classFromRomanBadInput(unittest.TestCase):
deftest_too_many_repeated_numerals(self):
'''from_romanshouldfailwithtoomanyrepeated
numerals'''
forsin('MMMM','DD','CCCC','LL','XXXX',
'VV','IIII'):
self.assertRaises(roman6.InvalidRomanNumeralError,
roman6.from_roman,s)
IX9
IXIX
deftest_repeated_pairs(self):
'''from_romanshouldfailwithrepeatedpairsof
numerals'''
forsin('CMCM','CDCD','XCXC','XLXL','IXIX',
'IVIV'):

CL150LC50
100
IMVX
deftest_malformed_antecedents(self):
'''from_romanshouldfailwithmalformed
antecedents'''
forsin('IIMXCC','VX','DCM','CMM','IXIV',
'MCMC','XCX','IVI','LM','LD',
'LC'):
from_roman()
InvalidRomanNumeralError
#roman6.py
classInvalidRomanNumeralError(ValueError):pass
from_roman()
romantest6.py
FFF.......
========================================================
==============
FAIL:test_malformed_antecedents
(__main__.FromRomanBadInput)
from_romanshouldfailwithmalformedantecedents
test_malformed_antecedents
AssertionError:InvalidRomanNumeralErrornotraisedby
from_roman
========================================================
==============
FAIL:test_repeated_pairs(__main__.FromRomanBadInput)
from_romanshouldfailwithrepeatedpairsofnumerals
File"romantest6.py",line107,intest_repeated_pairs
from_roman

========================================================
==============
FAIL:test_too_many_repeated_numerals
(__main__.FromRomanBadInput)
from_romanshouldfailwithtoomanyrepeatednumerals
test_too_many_repeated_numerals
from_roman
Ran10testsin0.058s
FAILED(failures=3)
from_roman()
roman_numeral_pattern=re.compile('''
^#beginningofstring
0300(0to3Cs),
#or500800(D,
followedby0to3Cs)
(XC|XL|L?X{0,3})#tens90(XC),40(XL),030
(0to3Xs),
#or5080(L,followed
by0to3Xs)
(IX|IV|V?I{0,3})#ones9(IX),4(IV),03(0
to3Is),
#or58(V,followedby
0to3Is)
$#endofstring
''',re.VERBOSE)
deffrom_roman(s):
ifnotroman_numeral_pattern.search(s):
raiseInvalidRomanNumeralError('InvalidRoman
numeral:{0}'.format(s))
result=0
index=0
result+=integer
index+=len(numeral)
returnresult
romantest7.py
..........
Ran10testsin0.066s
OK
OK
unittest
20019MarkPilgrim
:Python3
Afteronehasplayedavastquantityofnotesandmorenotes,itis
simplicitythatemergesasthecrowningrewardofart.
FrdricChopin
>>>importroman7
>>>roman7.from_roman('')
0
1.
InvalidRomanNumeralError
.
.
.
deftestBlank(self):
'''from_romanshouldfailwithblankstring'''
roman6.from_roman,'')
1.
from_roman()InvalidRomanNumeralError
romantest8.pyv
from_romanshouldfailwithblankstring...FAIL
from_romanshouldfailwithmalformedantecedents...ok
from_romanshouldfailwithrepeatedpairsof
numerals...ok
from_romanshouldfailwithtoomanyrepeated
numerals...ok
from_romanshouldgiveknownresultwithknowninput...
ok
ok
from_roman(to_roman(n))==nforalln...ok
========================================================
==============
FAIL:from_romanshouldfailwithblankstring
File"romantest8.py",line117,intest_blank
roman8.from_roman,'')
from_roman
Ran11testsin0.171s
FAILED(failures=1)

deffrom_roman(s):
ifnots:
raiseInvalidRomanNumeralError('Inputcannotbe
blank')
ifnotre.search(romanNumeralPattern,s):
numeral:{}'.format(s))
result=0
index=0
fornumeral,integerinromanNumeralMap:
result+=integer
index+=len(numeral)
returnresult
raise
2.
Python3.1
{0}format(){}
1.
Python
{}{0}{}{1}
romantest8.pyv
from_romanshouldfailwithblankstring...ok
numerals...ok
numerals...ok
ok
ok
Ran11testsin0.156s
OK
2.
1.
4M4000
1..39991..4999
[downloadroman8.py]
classKnownValues(unittest.TestCase):
known_values=((1,'I'),
.
.
.
(3999,'MMMCMXCIX'),
(4000,'MMMM'),
(4500,'MMMMD'),
(4888,'MMMMDCCCLXXXVIII'),
(4999,'MMMMCMXCIX'))
.
.
.
deftest_too_many_repeated_numerals(self):
'''from_romanshouldfailwithtoomanyrepeated
numerals'''
forsin('MMMMM','DD','CCCC','LL','XXXX',
'VV','IIII'):
.
.
.
classRoundtripCheck(unittest.TestCase):
deftest_roundtrip(self):
'''from_roman(to_roman(n))==nforalln'''

4000
4000()4500()4888()4999()
2.
4000to_roman()40004999
5000
3.
'MMMM'from_roman()MMMM
'MMMMM'
4.
13999
for4999
1.
romantest9.pyv
from_romanshouldfailwithnonstringinput...ok
numerals...ok
numerals...ok
ERROR
ERROR
from_roman(to_roman(n))==nforalln...ERROR
========================================================
==============
ERROR:from_romanshouldgiveknownresultwithknown
input
File"C:\home\diveintopython3\examples\roman9.py",
line60,infrom_roman
roman9.InvalidRomanNumeralError:InvalidRomannumeral:
MMMM
========================================================
==============
ERROR:to_romanshouldgiveknownresultwithknown
input
result=roman9.to_roman(integer)
line42,into_roman
raiseOutOfRangeError('numberoutofrange(mustbe
0..3999)')
roman9.OutOfRangeError:numberoutofrange(mustbe
0..3999)
========================================================
==============
ERROR:from_roman(to_roman(n))==nforalln
File"romantest9.py",line131,intestSanity
line42,into_roman
raiseOutOfRangeError('numberoutofrange(mustbe
0..3999)')
roman9.OutOfRangeError:numberoutofrange(mustbe
0..3999)
Ran12testsin0.171s
FAILED(errors=3)
'MMMM'from_roman()
from_roman()
2.
4000to_roman()
to_roman()
3.
4000to_roman()
1.
[downloadroman9.py]
roman_numeral_pattern=re.compile('''
^#beginningofstring
0300(0to3Cs),
#or500800(D,
followedby0to3Cs)
(XC|XL|L?X{0,3})#tens90(XC),40(XL),030
(0to3Xs),
#or5080(L,followed
by0to3Xs)
(IX|IV|V?I{0,3})#ones9(IX),4(IV),03(0
to3Is),
#or58(V,followedby
0to3Is)
$#endofstring
''',re.VERBOSE)
defto_roman(n):
ifnot(0<n<5000):
be1..4999)')
ifnotisinstance(n,int):
converted')
result=''
whilen>=integer:
result+=numeral
n=integer
returnresult
deffrom_roman(s):
.
.
.
from_roman()
roman_numeral_pattern
M34
49993999
from_roman()
'MMMM'
to_roman()
2.
0<n<40000<n<5000
(1..4999
1..3999)
'M'
4000'MMMM'
1.
romantest9.pyv
numerals...ok
numerals...ok
ok
ok
Ran12testsin0.203s
OK
1.
from_roman()
5000
[downloadroman10.py]
classOutOfRangeError(ValueError):pass
classNotIntegerError(ValueError):pass
classInvalidRomanNumeralError(ValueError):pass
roman_numeral_map=(('M',1000),
('CM',900),
('D',500),
('CD',400),
('C',100),
('XC',90),
('L',50),
('XL',40),
('X',10),
('IX',9),
('V',5),
('IV',4),
('I',1))
to_roman_table=[None]
from_roman_table={}
defto_roman(n):
ifnot(0<n<5000):
be1..4999)')
ifint(n)!=n:
converted')
returnto_roman_table[n]
deffrom_roman(s):
ifnotisinstance(s,str):
raiseInvalidRomanNumeralError('Inputmustbea
string')
ifnots:
blank')
ifsnotinfrom_roman_table:
returnfrom_roman_table[s]
defbuild_lookup_tables():
defto_roman(n):
result=''
ifn>=integer:
result=numeral
n=integer
break
ifn>0:
result+=to_roman_table[n]
returnresult
roman_numeral=to_roman(integer)
to_roman_table.append(roman_numeral)
from_roman_table[roman_numeral]=integer
build_lookup_tables()
if
if__name__=='__main__'
build_lookup_tables()?
to_roman_table=[None]
from_roman_table={}
.
.
.
defbuild_lookup_tables():
defto_roman(n):
result=''
ifn>=integer:
result=numeral
n=integer
break
ifn>0:
result+=to_roman_table[n]
returnresult
roman_numeral=to_roman(integer)
to_roman_table.append(roman_numeral)
from_roman_table[roman_numeral]=integer
1.
to_roman()
build_lookup_tables()to_roman()
build_lookup_tables()to_roman()
2.
to_roman()
3.
to_roman()
defto_roman(n):
ifnot(0<n<5000):
be1..4999)')
ifint(n)!=n:
converted')
returnto_roman_table[n]
deffrom_roman(s):
ifnotisinstance(s,str):
raiseInvalidRomanNumeralError('Inputmustbea
string')
ifnots:
blank')
ifsnotinfrom_roman_table:
returnfrom_roman_table[s]
to_roman()
2.
from_roman()
O(1)
1.
romantest10.pyv
numerals...ok
numerals...ok
ok
ok
Ran12testsin0.031s
OK
1.
10
Python
20019MarkPilgrim
Search
:
Aninemilewalkisnojoke,especiallyintherain.
HarryKemelman,TheNineMileWalk
Windows
38,493 Python3 3,000
Python
a_file=open('examples/chinese.txt',encoding='utf8')
Python open()
'examples/chinese.txt'
1.
2.
3.
4.
5.
open() Python
filename,
(forwardslash)
Windows
MacOSX Linux
Python Windows
(relative
path)
filename,
Windows Unicode
Python3 ASCII
(anentirelyvirtual
filesystem)
Python
open()filename
encoding
Unicode
Unicode
Python
Unicode
#ThisexamplewascreatedonWindows.Otherplatforms
may
#behavedifferently,forreasonsoutlinedbelow.
# Windows
>>>file=open('examples/chinese.txt')
>>>a_string=file.read()
File"C:\Python31\lib\encodings\cp1252.py",line23,
indecode
return
codecs.charmap_decode(input,self.errors,decoding_table)[
0]
UnicodeDecodeError:'charmap'codeccan'tdecodebyte
0x8finposition28:charactermapsto<undefined>
>>>
Python
(traceback) cp1252.py
Python CP1252
CP1252
CP1252
UnicodeDecodeError
(platformdependent)
UTF8
CP1252
locale
locale.getpreferredencoding()
Windows
'cp1252' Linux
'UTF8'
(consistency)
Windows
/
Python
open()open()streamobject
>>>a_file=open('examples/chinese.txt',encoding='utf
8')
>>>a_file.name
'examples/chinese.txt'
>>>a_file.encoding
'utf8'
>>>a_file.mode
'r'
1.
name open()
(normalize)
2.
encoding open()
encoding
locale.getpreferredencoding()
mode
3.
mode open()
Python 'r'
open()
>>>a_file=open('examples/chinese.txt',encoding='utf
8')
>>>a_file.read()
'DiveIntoPythonPython
\n'
>>>a_file.read()
''
read()
2.
Python (endoffile)
1.
encoding
#
>>>a_file.read()
''
>>>a_file.seek(0)
0
>>>a_file.read(16)
'DiveIntoPython'
>>>a_file.read(1)
''
>>>a_file.read(1)
''
>>>a_file.tell()
20
1.
2.
3.
4.
5.
read()
seek()
read()
16+1+1=20?
#
>>>a_file.seek(17)
17
>>>a_file.read(1)
''
>>>a_file.tell()
20
1.
2.
3.
17TH
20
seek()tell()
read()
UTF8
seek()read()
>>>a_file.seek(18)
18
>>>a_file.read(1)
File"<pyshell#12>",line1,in<module>
a_file.read(1)
File"C:\Python31\lib\codecs.py",line300,indecode
(result,consumed)=self._buffer_decode(data,
self.errors,final)
UnicodeDecodeError:'utf8'codeccan'tdecodebyte0x98
inposition0:unexpectedcodebyte
1.
2.
18TH
18
17
UnicodeDecodeError
#
>>>a_file.close()
(anticlimactic)
a_file close()
#
>>>a_file.read()
a_file.read()
ValueError:I/Ooperationonclosedfile.
>>>a_file.seek(0)
a_file.seek(0)
>>>a_file.tell()
a_file.tell()
>>>a_file.close()
>>>a_file.closed
True
1.
2.
3.
4.
5.
IOError
tell()
close()(noop)
closed
try..finally with
close()
close()
Python2 try..finally
Python3
Python3
Python2.5 Python3
with
withopen('examples/chinese.txt',encoding='utf8')as
a_file:
a_file.seek(17)
a_character=a_file.read(1)
print(a_character)
open()
a_file.close()with if
for a_file
open()
seek()read() with
Python a_file.close()
with
Python
exit
Python
with
(runtimecontext)
(contextmanager)Pythona_file
withPython
close()B
with
with
(genericframework)
with
(contextmanager)
ENTER
(endofaline)
(carriage
return)(linefeed)
Python
Python
(finegrained
control)
newlineopen()
open()
[downloadoneline.py]
line_number=0
withopen('examples/favoritepeople.txt',encoding='utf
8')asa_file:
fora_lineina_file:
line_number+=1
print('{:>4}{}'.format(line_number,
a_line.rstrip()))
withPython
2.
for
read()(iterator)
3.
format()
{:>4}
a_line
rstrip()
1.
you@localhost:~/diveintopython3$python3
examples/oneline.py
1Dora
2Ethan
3Wesley
4John
5Anne
6Mike
7Chris
8Sarah
9Alex
10Lizzie
you@localhost:~/diveintopython3$python3
examples/oneline.py
File"examples/oneline.py",line4,in
<module>
print('{:>4}{}'.format(line_number,
a_line.rstrip()))
ValueError:zerolengthfieldnameinformat
Python3.0
Python3.1
Python3.0
Python3.1
Python3.0
print('{0:>4}{1}'.format(line_number,
a_line.rstrip()))
open()
mode='w' open()
mode='a'
open()
(file
handle)
close() with
Python
>>>withopen('test.log',mode='w',encoding='utf8')as
a_file:
...a_file.write('testsucceeded')
>>>withopen('test.log',encoding='utf8')asa_file:
...print(a_file.read())
testsucceeded
>>>withopen('test.log',mode='a',encoding='utf8')as
a_file:
...a_file.write('andagain')
>>>withopen('test.log',encoding='utf8')asa_file:
...print(a_file.read())
testsucceededandagain
test.log
mode='w'
2.
open() write()
with Python
3.
with='a'
4.
test.log
'\n'
1.

open()
encoding
Python
Python
encoding
>>>an_image=open('examples/beauregard.jpg',mode='rb')
>>>an_image.mode
'rb'
>>>an_image.name
'examples/beauregard.jpg'
>>>an_image.encoding
AttributeError:'_io.BufferedReader'objecthasno
attribute'encoding'
mode 'b'
2.
mode open()
mode
3.
name
4.
encoding
Python
1.
#
>>>an_image.tell()
0
>>>data=an_image.read(3)
>>>data
b'\xff\xd8\xff'
>>>type(data)
<class'bytes'>
>>>an_image.tell()
3
>>>an_image.seek(0)
0
>>>data=an_image.read()
>>>len(data)
3150
&#hellip;
2.
&#hellip;
read()
3.
read()tell()
(unexpected
mismatch)
1.
read()
(library)
API
read()
size
size read()
size
read()
read()
>>>a_string='PapayaWhipisthenewblack.'
>>>importio
>>>a_file=io.StringIO(a_string)
>>>a_file.read()
'PapayaWhipisthenewblack.'
>>>a_file.read()
''
>>>a_file.seek(0)
0
>>>a_file.read(10)
'PapayaWhip'
>>>a_file.tell()
10
>>>a_file.seek(18)
18
>>>a_file.read()
'newblack.'
1.
2.
3.
4.
5.
6.
io StringIO
io.StringIO() StringIO
read() StringIO
read()
StringIO seek()
size read()
io.StringIO
io.ByteIO
Python
gzipbzip2Windows
gzip gzip
read() write()
gzip
with gzip
Python
you@localhost:~$python3
>>>importgzip
>>>withgzip.open('out.log.gz',mode='wb')asz_file:
...z_file.write('Aninemilewalkisnojoke,
especiallyintherain.'.encode('utf8'))
...
>>>exit()
you@localhost:~$lslout.log.gz
rwrr1markmark792009071914:29
out.log.gz
you@localhost:~$gunzipout.log.gz
you@localhost:~$catout.log
Aninemilewalkisnojoke,especiallyintherain.
gzip mode
'b'
2.
Linux
Pythonshell gzip
(longlistings) 79
gzip
(metadata)
gunzip geeunzip
3.
.gz
cat Python
4.
shell out.log.gz
1.
sys.stdin,sys.stdout,sys.stderr.
stdout stderr
UNIX (pipe) MacOSX
Linux print()
stdout
(traceback) stderr
(terminalwindow)
Pythonshell stdout stderr

(InteractiveWindow)
>>>foriinrange(3):
...print('PapayaWhip')
PapayaWhip
PapayaWhip
PapayaWhip
>>>importsys
>>>foriinrange(3):
...sys.stdout.write('isthe')
istheistheisthe
>>>foriinrange(3):
...sys.stderr.write('newblack')
newblacknewblacknewblack
print()
stdoutsys(stream
object)write()
print()
sys.stdout.write
3.
sys.stdout sys.stderr
PythonIDE
Python
1.
2.
read() IOError
>>>importsys
>>>sys.stdout.read()
IOError:notreadable
[downloadstdout.py]
importsys
classRedirectStdoutTo:
def__init__(self,out_new):
self.out_new=out_new
def__enter__(self):
self.out_old=sys.stdout
sys.stdout=self.out_new
def__exit__(self,*args):
sys.stdout=self.out_old
print('A')
withopen('out.log',mode='w',encoding='utf8')as
a_file,RedirectStdoutTo(a_file):
print('B')
print('C')
stdout.py
A
C
you@localhost:~/diveintopython3/examples$catout.log
B
you@localhost:~/diveintopython3/examples$
python3stdout.py
File"stdout.py",line15
withopen('out.log',mode='w',
encoding='utf8')asa_file,
RedirectStdoutTo(a_file):
^
SyntaxError:invalidsyntax
Python3.0
Python3.1
Python3.0 with
Python3.1 with
print('A')
print('B')
print('C')
with
a_file:
withRedirectStdoutTo(a_file):
print('B')
with
(scope)with
UTF8
out.log
a_file
withRedirectStdoutTo(a_file):
as (clause) with as
with
RedirectStdoutTo (sideeffect)
RedirectStdoutTo
(contextmanager)
code>__enter__()__exit__()
classRedirectStdoutTo:
def__init__(self,out_new):
self.out_new=out_new
def__enter__(self):
self.out_old=sys.stdout
sys.stdout=self.out_new
sys.stdout=self.out_old
__init__()
1.
(instancevariable)
__enter__()(specialclass
2.
method)Python
withsys.stdout
self.out_oldself.out_newsys.stdout
__exit__()
3.
with Python
self.out_old sys.stdout
print('A')
print('B')
print('C')
1.
IDE (Interactive
Window)
2.
with
with
sys.stdout
3.
print() with
out.log
4.
with Python
sys.stdout
out.log
print()
sys.stdout
sys.stderr
Python.org
io
sys.stdoutandsys.stderr
FUSE
20019MarkPilgrim
Search
:
XML
InthearchonshipofAristaechmus,Dracoenactedhisordinances.
Aristotle
XML
XML
(syndicationfeeds)
feed
(subscribe)feed
feed(feedaggregator)GoogleReader
XMLfeed
Atomfeed
[downloadfeed.xml]
<?xmlversion='1.0'encoding='utf8'?>
<feedxmlns='http://www.w3.org/2005/Atom'xml:lang='en'>
<title>diveintomark</title>
<subtitle>currentlybetweenaddictions</subtitle>
<id>tag:diveintomark.org,20010729:/</id>
<updated>20090327T21:56:07Z</updated>
<linkrel='alternate'type='text/html'
href='http://diveintomark.org/'/>
<linkrel='self'type='application/atom+xml'
href='http://diveintomark.org/feed/'/>
<entry>
<author>
<name>Mark</name>
<uri>http://diveintomark.org/</uri>
</author>
<title>Diveintohistory,2009edition</title>
href='http://diveintomark.org/archives/2009/03/27/dive
intohistory2009edition'/>
<id>tag:diveintomark.org,200903
27:/archives/20090327172042</id>
<published>20090327T17:20:42Z</published>
<categoryscheme='http://diveintomark.org'
term='diveintopython'/>
term='docbook'/>
term='html'/>
<summarytype='html'>Puttinganentirechapteronone
pagesounds
bloated,butconsiderthis—mylongest
chaptersofar
wouldbe75printedpages,anditloadsinunder5
seconds…
Ondialup.</summary>
</entry>
<entry>
<author>
<name>Mark</name>
</author>
<title>Accessibilityisaharshmistress</title>
href='http://diveintomark.org/archives/2009/03/21/access
ibilityisaharshmistress'/>
21:/archives/20090321200928</id>
term='accessibility'/>
<summarytype='html'>Theaccessibilityorthodoxy
doesnotpermitpeopleto
questionthevalueoffeaturesthatarerarely
usefulandrarelyused.</summary>
</entry>
<entry>
<author>
<name>Mark</name>
</author>
<title>Agentleintroductiontovideoencoding,part
1:containerformats</title>
href='http://diveintomark.org/archives/2008/12/18/give
part1containerformats'/>
18:/archives/20081218155422</id>
term='asf'/>
term='avi'/>
term='encoding'/>
term='flv'/>
term='GIVE'/>
term='mp4'/>
term='ogg'/>
term='video'/>
<summarytype='html'>Thesenoteswilleventually
becomepartofa
techtalkonvideoencoding.</summary>
</entry>
</feed>
5 XML
XML
XML XML
(tag)(element)
()XML
<foo>
</foo>
1.
2.
foo
foo
foo bar
<foo>
<bar></bar>
</foo>
XML (rootelement)
XML XML
<foo></foo>
<bar></bar>
(attribute)(namevalue)
<foolang='en'>
<barid='papayawhip'lang="fr"></bar>
</foo>
1.
2.
foo lang lang en

bar id lang lang
fr foo
Python
(textcontent)
<foolang='en'>
<barlang='fr'>PapayaWhip</bar>
</foo>
<foo></foo>
/
XML
<foo/>
Python (modules)
(namespace) XML XML
URL xmlns
<feedxmlns='http://www.w3.org/2005/Atom'>
</feed>
1.
2.
feed http://www.w3.org/2005/Atom
title
xmlns:prefix
prefix
(prefix)
<atom:feedxmlns:atom='http://www.w3.org/2005/Atom'>
<atom:title>diveintomark</atom:title>
</atom:feed>
1.
2.
feed http://www.w3.org/2005/Atom
title
XML XML
+=XML
(atom:)
XML
XML
(catch22)
XMLXML
XMLXMLF
XML
ATOM FEED
CNN.com(CNN.com)
(BreakingNews,U.S.,World,Weather,Entertainment&Video
News)(updated12:43p.m.EDT,SatMay
16,2009)
URL
Atom
CNN.com
CNN.com
Atom
http://www.w3.org/2005/Atom feed
<feedxmlns='http://www.w3.org/2005/Atom'
xml:lang='en'>
1.
2.
http://www.w3.org/2005/Atom Atom
xml:lang
xml:lang
feed
Atomfeed feed
feed diveintomark
currentlybetweenaddictions
feed(globallyunique
identifier)RFC4151
4.
feed March27,2009,at21:56
GMT
1.
2.
3.
5.
link
reltype hrefrel
rel='alternate' feed
type='text/html' HTML
href
feedonMarch27,2009
diveintomark
http://diveintomark.org/
XML
Atomfeed
feed
<entry>
<author>
<name>Mark</name>
</author>
<title>Diveintohistory,2009edition</title>
href='http://diveintomark.org/archives/2009/03/27/dive
intohistory2009edition'/>
27:/archives/20090327172042</id>
term='diveintopython'/>
term='docbook'/>
term='html'/>
<summarytype='html'>Puttinganentirechapteronone
pagesounds
bloated,butconsiderthis—mylongest
chaptersofar
wouldbe75printedpages,anditloadsinunder5
seconds…
Ondialup.</summary>
</entry>
1.
2.
3.
4.
5.
6.
7.
8.
author Mark
http://diveintomark.org/
feed
title Diveintohistory,2009
edition
feed link
HTML
feed
(published)
(updated)
diveintopythondocbook html
summary
content
summary Atom
type='html' HTML
HTML
—…
entry
XML
PythonXMLDOM
SAXElementTree
[downloadfeed.xml]
>>>importxml.etree.ElementTreeasetree
>>>tree=etree.parse('examples/feed.xml')
>>>root=tree.getroot()
>>>root
<Element{http://www.w3.org/2005/Atom}feedatcd1eb0>
ElementTree Python
xml.etree.ElementTree
parse()ElementTree
2.
parse()
XML
3.
parse()
getroot()
4.
http://www.w3.org/2005/Atom
feed
XML (localname)
Atom
{http://www.w3.org/2005/Atom}feed
1.
ElementTree
{namespace}localname XML
ElementTree API
ElementTreeAPI
>>>root.tag
'{http://www.w3.org/2005/Atom}feed'
>>>len(root)
8
>>>forchildinroot:
...print(child)
...
<Element{http://www.w3.org/2005/Atom}titleate2b5d0>
<Element{http://www.w3.org/2005/Atom}subtitleat
e2b4e0>
<Element{http://www.w3.org/2005/Atom}idate2b6c0>
<Element{http://www.w3.org/2005/Atom}updatedate2b6f0>
<Element{http://www.w3.org/2005/Atom}linkate2b4b0>
<Element{http://www.w3.org/2005/Atom}entryate2b720>
1.
{http://www.w3.org/2005/Atom}feed
2.
3.
4.
8 feed
titlesubtitleidupdated link
entry
entry
entry
feed
XML
Python
#continuingfromthepreviousexample
>>>root.attrib
{'{http://www.w3.org/XML/1998/namespace}lang':'en'}
>>>root[4]
<Element{http://www.w3.org/2005/Atom}linkate181b0>
>>>root[4].attrib
{'href':'http://diveintomark.org/',
'type':'text/html',
'rel':'alternate'}
>>>root[3]
<Element{http://www.w3.org/2005/Atom}updatedate2b4e0>
>>>root[3].attrib
{}
1.
2.
3.
4.
5.
attrib
<feed
xmlns='http://www.w3.org/2005/Atom'xml:lang='en'>
xml: XML
0 [4] link
link hreftype rel
[3] updated
updated .attrib
XML
XML
Etree
>>>root.findall('{http://www.w3.org/2005/Atom}entry')
[<Element{http://www.w3.org/2005/Atom}entryate2b4e0>,
<Element{http://www.w3.org/2005/Atom}entryate2b510>,
<Element{http://www.w3.org/2005/Atom}entryate2b540>]
>>>root.tag
'{http://www.w3.org/2005/Atom}feed'
>>>root.findall('{http://www.w3.org/2005/Atom}feed')
[]
>>>root.findall('{http://www.w3.org/2005/Atom}author')
[]
1.
findfall()
2.
findall()
feed feed
3.
authorentry
author
author
>>>tree.findall('{http://www.w3.org/2005/Atom}entry')
>>>tree.findall('{http://www.w3.org/2005/Atom}author')
[]
tree etree.parse()
tree.getroot().findall()
2.
author
1.
tree.getroot().findall('{http://www.w3.org/2005/Atom}aut
hor')
author author entry
find()
>>>entries=
tree.findall('{http://www.w3.org/2005/Atom}entry')
>>>len(entries)
3
>>>title_element=
entries[0].find('{http://www.w3.org/2005/Atom}title')
>>>title_element.text
'Diveintohistory,2009edition'
>>>foo_element=
entries[0].find('{http://www.w3.org/2005/Atom}foo')
>>>foo_element
>>>type(foo_element)
<class'NoneType'>
atom:entry
2.
find() ElementTree
3.
entries[0] foo None
1.
find()
ElementTree
False len(element)
0 if
element.find('...')
find()
find()
ifelement.find('...')isnot
None
(descendant)
>>>all_links=
tree.findall('//{http://www.w3.org/2005/Atom}link')
>>>all_links
[<Element{http://www.w3.org/2005/Atom}linkate181b0>,
<Element{http://www.w3.org/2005/Atom}linkate2b570>,
<Element{http://www.w3.org/2005/Atom}linkate2b480>,
<Element{http://www.w3.org/2005/Atom}linkate2b5a0>]
>>>all_links[0].attrib
{'href':'http://diveintomark.org/',
'type':'text/html',
'rel':'alternate'}
{'href':
'http://diveintomark.org/archives/2009/03/27/diveinto
history2009edition',
'type':'text/html',
'rel':'alternate'}
{'href':
'http://diveintomark.org/archives/2009/03/21/accessibili
tyisaharshmistress',
'type':'text/html',
'rel':'alternate'}
{'href':
'http://diveintomark.org/archives/2008/12/18/givepart
1containerformats',
'type':'text/html',
'rel':'alternate'}
1.
//{http://www.w3.org/2005/Atom}link
findall()
2.
feed HTML
3.
entry
link
ElementTreefindall()
XPathXPathXMLW3C
ElementTreeXPath
XPath
XML
ElementTreeAPIXPath
LXML
lxmllibxml2
ElementTreeAPI
XPath1.0
WindowsLinux
yumaptget
>>>fromlxmlimportetree
>>>root.findall('{http://www.w3.org/2005/Atom}entry')
lxml ElementTree
API
2.
parse() ElementTree
getroot()
3.
4.
findall()
1.
XML lxml ElementTree

ElementTree API
(implementation) lxml
ElementTree
try:
fromlxmlimportetree
exceptImportError:
importxml.etree.ElementTreeasetree
lxml ElementTree findall()
>>>importlxml.etree
>>>tree=lxml.etree.parse('examples/feed.xml')
>>>
tree.findall('//{http://www.w3.org/2005/Atom}*[@href]')
[<Element{http://www.w3.org/2005/Atom}linkateeb8a0>,
<Element{http://www.w3.org/2005/Atom}linkateeb990>,
<Element{http://www.w3.org/2005/Atom}linkateeb960>,
<Element{http://www.w3.org/2005/Atom}linkateeb9c0>]
>>>
tree.findall("//{http://www.w3.org/2005/Atom}*[@href='ht
tp://diveintomark.org/']")
[<Element{http://www.w3.org/2005/Atom}linkateeb930>]
>>>NS='{http://www.w3.org/2005/Atom}'
>>>tree.findall('//{NS}author[{NS}uri]'.format(NS=NS))
[<Element {http://www.w3.org/2005/Atom}author at eeba80>,

<Element {http://www.w3.org/2005/Atom}author at eebba0>]
importlxml.etree from
lxmlimportetree lxml
2.
Atom href
//
{http://www.w3.org/2005/Atom}
Atom *(localname)[@href]
href
3.
href
http://diveintomark.org/ Atom
4.
Atomuri
authorentry
authorentryauthorname
uri
1.
lxml XPath1.0
XPath
lxml
>>>importlxml.etree
>>>tree=lxml.etree.parse('examples/feed.xml')
>>>NSMAP={'atom':'http://www.w3.org/2005/Atom'}
>>>entries=
tree.xpath("//atom:category[@term='accessibility']/..",
...namespaces=NSMAP)
>>>entries
[<Element{http://www.w3.org/2005/Atom}entryate2b630>]
>>>entry=entries[0]
>>>entry.xpath('./atom:title/text()',namespaces=NSMAP)
['Accessibilityisaharshmistress']
Python
2.
XPath XPath
category accessibility
term
/..
category XPath
<categoryterm='accessibility'>
3.
xpath() ElementTree
category term
accessibility
4.
XPath
XML DOM
(node)
XPath
title (atom:title)
(text()) title (./)
1.
XML
Python XML
XML
>>>new_feed=
etree.Element('{http://www.w3.org/2005/Atom}feed',
...
attrib={'{http://www.w3.org/XML/1998/namespace}lang':
'en'})
>>>print(etree.tostring(new_feed))
<ns0:feedxmlns:ns0='http://www.w3.org/2005/Atom'
xml:lang='en'/>
Element
+ Atom
feed
2.
attrib
ElementTree {namespace}localname
3.
ElementTree tostring()
1.
ElementTree
XML (default
namespace)(xmlns='http://www.w3.org/2005/Atom')
Atomfeeds
(<feed>
<link><entry>)
XML
XML
DOM
<ns0:feedxmlns:ns0='http://www.w3.org/2005/Atom'
xml:lang='en'/>
DOM
xml:lang='en'/>
feedns0:
479+4
320
UTF8 320 gzip
21 21
Atomfeed
ElementTree
lxml
>>>importlxml.etree
>>>NSMAP={None:'http://www.w3.org/2005/Atom'}
>>>new_feed=lxml.etree.Element('feed',nsmap=NSMAP)
>>>print(lxml.etree.tounicode(new_feed))
<feedxmlns='http://www.w3.org/2005/Atom'/>
>>>
new_feed.set('{http://www.w3.org/XML/1998/namespace}lang
','en')
xml:lang='en'/>
None
2.
lxml nsmap
lxml
3.
Atom
feed
4.
xml:lang set()
1.
ElementTree
lxml nsmap lxml
XML
>>>title=lxml.etree.SubElement(new_feed,'title',
...attrib={'type':'html'})
xml:lang='en'><titletype='html'/></feed>
>>>title.text='diveinto…'
xml:lang='en'><titletype='html'>diveinto
…</title></feed>
>>>print(lxml.etree.tounicode(new_feed,
pretty_print=True))
<titletype='html'>diveinto…</title>
</feed>
1.
SubElement
new_feed
2.
3.
title Atom
feed title
lxml
/>
4.
.text
5.
title
<&lxml
6.
(pretty
printing)
lxml
(insignificantwhitespace)
xmlwitch
XML
withXML
XML
XMLXML
(draconian)XML
(wellformedness)
(entity)Unicode
(esotericrules)
HTMLHTML
&
WebHTML
HTML
XML
XML Atomfeeds
HTTP 1997 XML
XML
Atomfeeds
XML
lxml
XML
<title>diveinto…</title>
...
</feed>
… XML
HTML
feedlxml
>>>importlxml.etree
>>>tree=lxml.etree.parse('examples/feedbroken.xml')
File"lxml.etree.pyx",line2693,inlxml.etree.parse
(src/lxml/lxml.etree.c:52591)
File"parser.pxi",line1478,in
lxml.etree._parseDocument(src/lxml/lxml.etree.c:75665)
lxml.etree._parseDocumentFromURL
lxml.etree._parseDocFromFile
lxml.etree._BaseParser._parseDocFromFile
lxml.etree._ParserContext._handleParseResultDoc
lxml.etree._handleParseResult
lxml.etree._raiseParseError (src/lxml/lxml.etree.c:68125)
lxml.etree.XMLSyntaxError:Entity'hellip'notdefined,
line3,column28
XML
XML
>>>parser=lxml.etree.XMLParser(recover=True)
>>>tree=lxml.etree.parse('examples/feedbroken.xml',
parser)
>>>parser.error_log
examples/feed
broken.xml:3:28:FATAL:PARSER:ERR_UNDECLARED_ENTITY:
Entity'hellip'notdefined
>>>tree.findall('{http://www.w3.org/2005/Atom}title')
[<Element{http://www.w3.org/2005/Atom}titleatead510>]
>>>title=
tree.findall('{http://www.w3.org/2005/Atom}title')[0]
>>>title.text
'diveinto'
>>>print(lxml.etree.tounicode(tree.getroot()))
<title>diveinto</title>
.
.[restofserializationsnippedforbrevity]
.
1.
lxml.etree.XMLParser
recoverTrueXML
2.
XML parser
parse()lxml
…
3.
4.
…
title 'diveinto'
5.
…
XML
(interoperability)
… HTML…
XML
XML
ElementTreeXMLAPI
ElementTreeXPath
ElementTree(iterparse)
lxml
lxmlXMLHTMLwith
lxmlXPathXSLT
xmlwitch
20019MarkPilgrim
Search
:Python3
:
PYTHON
EverySaturdaysincewevelivedinthisapartment,Ihave
awakenedat6:15,pouredmyselfabowlofcereal,added
aquartercupof2%milk,satonthisendofthiscouch,turnedon
BBCAmerica,andwatchedDoctorWho.
Sheldon,TheBigBangTheory
(,),
pickle Python
,; Python
C Python
pickle ?
Python:,,,,
,bytes(),,None.
(Python
).
()
pickle
PythonShell
picklejson
PythonShell
PythonShell:
>>>shell=1
PythonShell
:
>>>shell=2
, shell
PythonShell
PICKLE
pickle
>>>shell
>>>entry={}
>>>entry['title']='Diveintohistory,2009edition'
>>>entry['article_link']=
history2009edition'
>>>entry['comments_link']=None
>>>entry['internal_id']=b'\xDE\xD5\xB4\xF8'
>>>entry['tags']=('diveintopython','docbook','html')
>>>entry['published']=True
>>>importtime
>>>entry['published_date']=time.strptime('FriMar27
22:20:422009')
>>>entry['published_date']
time.struct_time(tm_year=2009,tm_mon=3,tm_mday=27,
tm_hour=22,tm_min=20,tm_sec=42,tm_wday=4,tm_yday=86,
tm_isdst=1)
1.
PythonShell#1
Python
Atomentrypickle
3.
time( 1 )
(time_struct)strptime()
time_struct
time
2.
Python
>>>shell
1
>>>importpickle
>>>withopen('entry.pickle','wb')asf:
...pickle.dump(entry,f)
...
PythonShell#1
open()'wb'
with
pickle dump() Python

3.
, pickle
Python
1.
2.
pickle Python
pickle
pickle Python
Perl,PHP,Java,
entry.pickle
Python pickle
Python pickle
Python
Python Python
()
pickle pickle
pickle
Python
pickle
pickle,
PICKLE
PythonShell entry
>>>shell
2
>>>entry
NameError:name'entry'isnotdefined
>>>importpickle
>>>withopen('entry.pickle','rb')asf:
...entry=pickle.load(f)
...
>>>entry
{'comments_link':None,
'internal_id':b'\xDE\xD5\xB4\xF8',
'title':'Diveintohistory,2009edition',
'tags':('diveintopython','docbook','html'),
'article_link':
'published_date':time.struct_time(tm_year=2009,
tm_mon=3,tm_mday=27,tm_hour=22,tm_min=20,tm_sec=42,
tm_wday=4,tm_yday=86,tm_isdst=1),
'published':True}
PythonShell#2.
entry PythonShell#1
entry ,
3.
PythonShell#1 entry.pickle
pickle
pickle
4.
pickle.load(),
PythonPython
Python
5.
entry
1.
2.
pickle.dump()/pickle.load()
>>>shell
1
...entry2=pickle.load(f)
...
>>>entry2==entry
True
>>>entry2isentry
False
>>>entry2['tags']
('diveintopython','docbook','html')
>>>entry2['internal_id']
b'\xDE\xD5\xB4\xF8'
PythonShell#1
entry.pickle
,entry2
Python,entryentry2
shell , entry,
entry.pickle
5.
1.
2.
3.
4.
6.
'tags''internal_id'
bytes
Python
bytes
>>>shell
1
>>>b=pickle.dumps(entry)
>>>type(b)
<class'bytes'>
>>>entry3=pickle.loads(b)
>>>entry3==entry
True
1.
pickle.dumps()('s')
pickle.dump()
2.
pickle
pickle.dumps() bytes
pickle.loads()(,'s')
3.
pickle.load()
bytes , pickle.dumps()
4.
picklePython
pickle
Python1.x pickle (0)

(1).
Python2.3 pickle (2)
Python
Python3.0 pickle(3)
bytes
,(
),
Python32pickle,Python2
3 .
PICKLE
pickle PythonShell
entry.pickle
you@localhost:~/diveintopython3/examples$lsl
entry.pickle
rwrr1youyou358Aug313:34entry.pickle
you@localhost:~/diveintopython3/examples$cat
entry.pickle
comments_linkqNXtagsqXdiveintopythonqXdocbookqXhtmlq?qX
publishedq?
XlinkXJhttp://diveintomark.org/archives/2009/03/27/dive
intohistory2009edition
qXpublished_dateq
ctime
struct_time
?qRqXtitleqXDiveintohistory,2009editionqu.
()
()
>>>shell
1
>>>importpickletools
...pickletools.dis(f)
0:\x80PROTO3
2:}EMPTY_DICT
3:qBINPUT0
5:(MARK
6:XBINUNICODE'published_date'
25:qBINPUT1
27:cGLOBAL'timestruct_time'
45:qBINPUT2
47:(MARK
48:MBININT22009
51:KBININT13
53:KBININT127
55:KBININT122
57:KBININT120
59:KBININT142
61:KBININT14
63:KBININT186
65:JBININT1
70:tTUPLE(MARKat47)
71:qBINPUT3
73:}EMPTY_DICT
74:qBINPUT4
76:\x86TUPLE2
77:qBINPUT5
79:RREDUCE
80:qBINPUT6
82:XBINUNICODE'comments_link'
100:qBINPUT7
102:NNONE
103:XBINUNICODE'internal_id'
119:qBINPUT8
121:CSHORT_BINBYTES''
127:qBINPUT9
129:XBINUNICODE'tags'
138:qBINPUT10
140:XBINUNICODE'diveintopython'
159:qBINPUT11
161:XBINUNICODE'docbook'
173:qBINPUT12
175:XBINUNICODE'html'
184:qBINPUT13
186:\x87TUPLE3
187:qBINPUT14
189:XBINUNICODE'title'
199:qBINPUT15
201:XBINUNICODE'Diveintohistory,2009
edition'
237:qBINPUT16
239:XBINUNICODE'article_link'
256:qBINPUT17
258:XBINUNICODE
history2009edition'
337:qBINPUT18
339:XBINUNICODE'published'
353:qBINPUT19
355:\x88NEWTRUE
356:uSETITEMS(MARKat5)
357:.STOP
highestprotocolamongopcodes=3
,
pickle pickle
pickle
(opcodes)
()
pickle.dis()
:
[pickleversion.py]
importpickletools
defprotocol_version(file_object):
maxproto=1
foropcode,arg,posin
pickletools.genops(file_object):
maxproto=max(maxproto,opcode.proto)
returnmaxproto
:
>>>importpickleversion
...v=pickleversion.protocol_version(f)
>>>v
PYTHON
picklePython
JSON
JSONJavaScriptObjectNotation,
JSON
Python3jsonpickle,
json
Pythone,
JSON,RFC4627
JSON
5 'false' 4
'true'JSON
,(whitespaces)JSON
(,,)
JSON
, JSON
prettyprintJSON,
Pythonjson
prettyprinting
,JSON,
,JSONUnicode
(UTF32,UTF16,,UTF8),RFC4627
3
JSON
JSONJavascript
;JavaScripteval()
JSON(,
JSONJavaScript),
JSON
>>>shell
1
>>>basic_entry={}
>>>basic_entry['id']=256
>>>basic_entry['title']='Diveintohistory,2009
edition'
>>>basic_entry['tags']=('diveintopython','docbook',
'html')
>>>basic_entry['published']=True
>>>basic_entry['comments_link']=None
>>>importjson
>>>withopen('basic.json',mode='w',encoding='utf8')
asf:
...json.dump(basic_entry,f)
entry
, JSON
2.
JSON
UTF8
3.
pickle ,json dump()
Pythondump()
Python with
1.
JSON
you@localhost:~/diveintopython3/examples$catbasic.json
{"published":true,"tags":["diveintopython","docbook",
"html"],"comments_link":null,
"id":256,"title":"Diveintohistory,2009edition"}
pickleJSON
,json
JSON
>>>shell
1
>>>withopen('basicpretty.json',mode='w',
encoding='utf8')asf:
...json.dump(basic_entry,f,indent=2)
1.
json.dump() indent ,
JSON indent0
0
:
you@localhost:~/diveintopython3/examples$catbasic
pretty.json
{
"published":true,
"tags":[
"diveintopython",
"docbook",
"html"
],
"comments_link":null,
"id":256,
"title":"Diveintohistory,2009edition"
}
PYTHONJSON
JSON Python Python
Python :
JSON
object
array
string
integer
realnumber
true
Python3
dictionary
list
string
integer
float
True
false
*
null
*
*JSON
False
None
&bytes!JSON
,json Python ,
(frozenarrays)()JSON
bytes
JSON
JSON ,
bytes json
(JSON json
, JSON )
JSON
>>>shell
1
>>>entry
'tags':('diveintopython','docbook','html'),
'article_link':
'published':True}
>>>importjson
>>>withopen('entry.json','w',encoding='utf8')asf:
...json.dump(entry,f)
...
File"C:\Python31\lib\json\__init__.py",line178,in
dump
forchunkiniterable:
File"C:\Python31\lib\json\encoder.py",line408,in
_iterencode
forchunkin_iterencode_dict(o,
_current_indent_level):
_iterencode_dict
forchunkinchunks:
_iterencode
o=_default(o)
default
raiseTypeError(repr(o)+"isnotJSON
serializable")
TypeError:b'\xDE\xD5\xB4\xF8'isnotJSONserializable
, entry:
None ,bytes , time
2.
JSON
UTF8
JSON
3.
1.
:json.dump() bytes
b'\xDE\xD5\xB4\xF8' JSON
bytes ,
[downloadcustomserializer.py]
defto_json(python_object):
ifisinstance(python_object,bytes):
return{'__class__':'bytes',
'__value__':list(python_object)}
raiseTypeError(repr(python_object)+'isnotJSON
serializable')
JSON
, Python
json.dump()
bytesb'\xDE\xD5\xB4\xF8'
2.
json.dump()
3.
, bytes__class__
(,'bytes'),__value__
bytes ;
JSON !bytes
0255 list()
bytes b'\xDE\xD5\xB4\xF8'
[222,213,180,248].(!!16 \xDE
222,\xD5213,)
4.
JSON
TypeError
json.dump()
1.
,
Python
JSON ;
json.dump()
>>>shell
>>>importcustomserializer
...json.dump(entry,f,
default=customserializer.to_json)
...
json.dump(entry, f, default=customserializer.to_json)
File"C:\Python31\lib\json\__init__.py",line178,in
dump
forchunkiniterable:
_iterencode
forchunkin_iterencode_dict(o,
_current_indent_level):
_iterencode_dict
forchunkinchunks:
_iterencode
o=_default(o)
File
"/Users/pilgrim/diveintopython3/examples/customserialize
r.py",line12,into_json
serializable')
TypeError:time.struct_time(tm_year=2009,tm_mon=3,
tm_mday=27,tm_hour=22,tm_min=20,tm_sec=42,tm_wday=4,
tm_yday=86,tm_isdst=1)isnotJSONserializable
1.
customserializer to_json()
2.
,UTF8,yaddayadda(!
!,
)
3.
:json.dump(),
defaultjson.dump()(,
Python!)
4.
,json.dump()
bytes
:time.struct_time
,
!
importtime
defto_json(python_object):
ifisinstance(python_object,time.struct_time):
return{'__class__':'time.asctime',
'__value__': time.asctime(python_object)}
ifisinstance(python_object,bytes):
return{'__class__':'bytes',
'__value__':list(python_object)}
serializable')
customserializer.to_json(),
Python(json.dump())
time.struct_time
2.
bytes
: time.struct_time JSON
, JSON
time.asctime()
time.asctime() time.struct_time
'FriMar2722:20:422009'
1.
, entry JSON
>>>shell
1
...json.dump(entry,f,
default=customserializer.to_json)
...
you@localhost:~/diveintopython3/examples$lsl
example.json
rwrr1youyou391Aug313:34entry.json
you@localhost:~/diveintopython3/examples$cat
example.json
{"published_date":{"__class__":"time.asctime",
"__value__":"FriMar2722:20:422009"},
"comments_link":null,"internal_id":{"__class__":
"bytes","__value__":[222,213,180,248]},
"tags":["diveintopython","docbook","html"],"title":
"Diveintohistory,2009edition",
"article_link":
"http://diveintomark.org/archives/2009/03/27/diveinto
history2009edition",
"published":true}
JSON
picklejson load()
JSON , JSON
Python
>>>shell
2
>>>delentry
>>>entry
NameError:name'entry'isnotdefined
>>>importjson
>>>withopen('entry.json','r',encoding='utf8')asf:
...entry=json.load(f)
...
>>>entry
'internal_id':{'__class__':'bytes','__value__':[222,
213,180,248]},
'tags':['diveintopython','docbook','html'],
'article_link':
'published_date':{'__class__':'time.asctime',
'__value__':'FriMar2722:20:422009'},
'published':True}
PythonShell#2
pickle entry
2.
json.load() pickle.load()
Python
3.
:json.load()
PythonShell#1 entry.json
Python :
entry'internal_id'
'published_date',
to_json() JSON
1.
json.load() json.dump()
to_json()
JSON Python
#addthistocustomserializer.py
deffrom_json(json_object):
if'__class__'injson_object:
ifjson_object['__class__']=='time.asctime':
return
time.strptime(json_object['__value__'])
ifjson_object['__class__']=='bytes':
returnbytes(json_object['__value__'])
returnjson_object
Python JSON
Python
2.
to_json()
'__class__''__class__'
Python
3.
time.asctime()
time.strptime()
( time.asctime()
) time.struct_time.
4.
bytes,bytes()
1.
;to_json()
from_json():
>>>shell
2
...entry=json.load(f,
object_hook=customserializer.from_json)
...
>>>entry
'tags':['diveintopython','docbook','html'],
'article_link':
'published':True}
from_json()
object_hook json.load()
;!
entry bytes 'internal_id'
2.
'published_date'
time.struct_time
1.
>>>shell
1
...entry2=json.load(f,
object_hook=customserializer.from_json)
...
>>>entry2==entry
False
>>>entry['tags']
('diveintopython','docbook','html')
>>>entry2['tags']
['diveintopython','docbook','html']
to_json(),
from_json(),
2.
entry,'tags'
3.
entry2,'tags'
JSON
json
JSON
json
1.
pickle
cPicklePython2 ,pickle
,PythonC
(Python)
Python3 ,,
importpickle
cPickle.
pickle :
picklemodule
pickleandcPicklePythonobjectserialization
Usingpickle
Pythonpersistencemanagement
JSONjson:
jsonJavaScriptObjectNotationSerializer
JSONencodingandecodingwithcustomobjectsinPython
:
Picklingclassinstances
Persistenceofexternalobjects
Handlingstatefulobjects
20019MarkPilgrim
Search
:Python3
:
HTTP WEB
Aruffledmindmakesarestlesspillow.
CharlotteBront
HTTPwebHTTP
HTTPGET
HTTPPOST. HTTPWebAPI HTTP
PUTHTTPDELETE
HTTPverbs()(GET,POST,PUTDELETE)
,
XML JSON
,(
Python) HTTP;
HTTPweb( URL
),.
HTTPweb:
GoogleDataAPIGoogle,
BloggerYouTube
FlickrServicesFlickr
TwitterAPITwitter
Python3 HTTPweb:
http.clientRFC2616,HTTP.
urllib.requesthttp.client
HTTPFTPAPI
HTTPHTTP
,
httplib2,,http.client
HTTPurllib.request
httplib2 HTTP
HTTP
HTTP
web
()
(neveradull
moment)CacheControl:maxage
HTTP (
)
ISP
,
HTTP
diveintomark.org
wearehugh.com/m.jpg,
HTTP:
HTTP/1.1200OK
Date:Sun,31May200917:14:04GMT
Server:Apache
LastModified:Fri,22Aug200804:28:16GMT
ETag:"3075ddc8d800"
AcceptRanges:bytes
ContentLength:12405
CacheControl:maxage=31536000,public
Expires:Mon,31May201017:14:04GMT
Connection:close
ContentType:image/jpeg
CacheControlExpires(
)
!
.
HTTP
(CacheControl
public )
ISP
diveintomark.org
,,,
;,
()(
)
HTTP
Python HTTP httplib2
CNN.com
(
cache"")
1
!304:NotModified
(sameshit,differentday)
HTTP
LastModified
diveintomark.org
LastModified
HTTP/1.1200OK
Server:Apache
ETag:"3075ddc8d800"
AcceptRanges:bytes
ContentLength:12405
Connection:close
(
IfModifiedSince
IfModifiedSince 200
HTTP304,
.
curl:
you@localhost:~$curlIH"IfModifiedSince:Fri,22
Aug200804:28:16GMT"http://wearehugh.com/m.jpg
HTTP/1.1304NotModified
Server:Apache
Connection:close
ETag:"3075ddc8d800"
304 ,
( 304
;
304
)
ETAGS
ETag
ETagETag
(
)diveintomark.orgETag
.
HTTP/1.1200OK
Server:Apache
ETag:"3075ddc8d800"
AcceptRanges:bytes
ContentLength:12405
Connection:close
ETag
IfNoneMatchETag
304
304
ETag
.
curl:
you@localhost:~$curlIH"IfNoneMatch:\"3075
ddc8d800\""http://wearehugh.com/m.jpg
HTTP/1.1304NotModified
Server:Apache
Connection:close
ETag:"3075ddc8d800"
1.
ETag,
ETag ETag"3075ddc8d800"
IfNoneMatch
PythonHTTP ETag httplib2 .
HTTPweb,
XMLJSON
XML
3070gzip
941 30%!
HTTPgzipdeflate
HTTP
Acceptencoding
(Contentencoding
)
URIURI
web
http://example.com/index.xml
http://example.com/xml/atom.xml
;
http://www.example.com/index.xmlhttp://serverfarm
1.example.com/index.xml.Location!
HTTP ,
200
; 404 ;(
404)300
HTTP
302301302;,
;( Location
) 301 ;
;( Location )
302 ,HTTP
301 ,
urllib.request HTTP
urllib.request
httplib2
URL
HTTP
HTTP,
Atom,
()
>>>importurllib.request
>>>a_url=
'http://diveintopython3.org/examples/feed.xml'
>>>data=urllib.request.urlopen(a_url).read()
>>>type(data)
<class'bytes'>
>>>print(data)
Python HTTP ;
urllib.request
urlopen()
read()
urlopen().read()bytes,
2.
HTTP
1.
Web ( 1
),
Python
HTTP (
).
>>>fromhttp.clientimportHTTPConnection
>>>HTTPConnection.debuglevel=1
>>>fromurllib.requestimporturlopen
>>>response=
urlopen('http://diveintopython3.org/examples/feed.xml')
send:b'GET/examples/feed.xmlHTTP/1.1
Host:diveintopython3.org
AcceptEncoding:identity
UserAgent:Pythonurllib/3.1'
Connection:close
reply:'HTTP/1.1200OK'
furtherdebugginginformationomitted
1.
urllib.request
Python ,http.client
http.client(urllib.request)
HTTPConnection
urllib.request HTTP
2.
3.
4.
5.
6.
HTTP
Atom,
urllib.request 5
HTTP (
)
urllib.request
Python
urllib urllib.request httplib2
, UserAgent (
).
3070 941 .
#continuedfrompreviousexample
>>>print(response.headers.as_string())
Server:Apache
LastModified:Sun,31May200906:39:55GMT
ETag:"bfe93d9c4c0"
AcceptRanges:bytes
ContentLength:3070
CacheControl:maxage=86400
Expires:Mon,01Jun200919:23:06GMT
Vary:AcceptEncoding
Connection:close
ContentType:application/xml
>>>data=response.read()
>>>len(data)
3070
1.
2.
3.
4.
5.
6.
7.
urllib.request.urlopen() response
HTTP
LastModified
ETag
3070 :
Contentencoding
(Acceptencoding:identity),
24
(86400).
response.read(). len()
3070
()
gzip,
HTTP
941 3070
Baddog,nobiscuit.
>>>response2=
urlopen('http://diveintopython3.org/examples/feed.xml')
UserAgent:Pythonurllib/3.1'
Connection:close
IfModifiedSince.IfNoneMatch
.
>>>print(response2.headers.as_string())
Date:Mon,01Jun200903:58:00GMT
Server:Apache
LastModified:Sun,31May200922:51:11GMT
ETag:"bfe255ef5c0"
AcceptRanges:bytes
ContentLength:3070
CacheControl:maxage=86400
Expires:Tue,02Jun200903:58:00GMT
Vary:AcceptEncoding
Connection:close
ContentType:application/xml
>>>data2=response2.read()
>>>len(data2)
3070
>>>data2==data
True
:CacheControl
Expires,LastModifiedETag
Vary:AcceptEncoding
2.
3070
3.
3070
1.
HTTPurllib HTTP
HTTP
HTTP
HTTPLIB2
httplib2 ,
code.google.com/p/httplib2/httplib2
Python2.xPython3.x;
Python3,httplib2python30.5.0.zip
, httplib2
Windows,, cmd.exe
(ENTER).
c:\Users\pilgrim\Downloads>dir
VolumeindriveChasnolabel.
VolumeSerialNumberisDED5B4F8
Directoryofc:\Users\pilgrim\Downloads
07/28/200912:36PM<DIR>.
07/28/200912:36PM<DIR>..
07/28/200912:36PM<DIR>httplib2python3
0.5.0
07/28/200912:33PM18,997httplib2python3
0.5.0.zip
1File(s)18,997bytes
3Dir(s)61,496,684,544bytesfree
c:\Users\pilgrim\Downloads>cdhttplib2python30.5.0
c:\Users\pilgrim\Downloads\httplib2python30.5.0>
c:\python31\python.exesetup.pyinstall
runninginstall
runningbuild
runningbuild_py
runninginstall_lib
creatingc:\python31\Lib\sitepackages\httplib2
copyingbuild\lib\httplib2\iri2uri.py>
c:\python31\Lib\sitepackages\httplib2
copyingbuild\lib\httplib2\__init__.py>
c:\python31\Lib\sitepackages\httplib2
bytecompilingc:\python31\Lib\site
packages\httplib2\iri2uri.pytoiri2uri.pyc
bytecompilingc:\python31\Lib\site
packages\httplib2\__init__.pyto__init__.pyc
runninginstall_egg_info
Writingc:\python31\Lib\sitepackages\httplib2
python3_0.5.0py3.1.egginfo
MacOSX ,/Applications/Utilities/
Terminal.app Linux (Terminal),
Accessories
(System)
you@localhost:~/Desktop$unziphttplib2python3
0.5.0.zip
Archive:httplib2python30.5.0.zip
inflating:httplib2python30.5.0/README
inflating:httplib2python30.5.0/setup.py
inflating:httplib2python30.5.0/PKGINFO
inflating:httplib2python30.5.0/httplib2/__init__.py
inflating:httplib2python30.5.0/httplib2/iri2uri.py
you@localhost:~/Desktop$cdhttplib2python30.5.0/
you@localhost:~/Desktop/httplib2python30.5.0$sudo
python3setup.pyinstall
runninginstall
runningbuild
runningbuild_py
creatingbuild
creatingbuild/lib.linuxx86_643.1
creatingbuild/lib.linuxx86_643.1/httplib2
copyinghttplib2/iri2uri.py>build/lib.linuxx86_64
3.1/httplib2
copyinghttplib2/__init__.py>build/lib.linuxx86_64
3.1/httplib2
runninginstall_lib
creating/usr/local/lib/python3.1/distpackages/httplib2
copyingbuild/lib.linuxx86_643.1/httplib2/iri2uri.py
>/usr/local/lib/python3.1/distpackages/httplib2
copyingbuild/lib.linuxx86_643.1/httplib2/__init__.py
>/usr/local/lib/python3.1/distpackages/httplib2
bytecompiling/usr/local/lib/python3.1/dist
packages/httplib2/iri2uri.pytoiri2uri.pyc
bytecompiling/usr/local/lib/python3.1/dist
packages/httplib2/__init__.pyto__init__.pyc
Writing/usr/local/lib/python3.1/distpackages/httplib2
python3_0.5.0.egginfo
httplib2, httplib2.Http
>>>importhttplib2
>>>h=httplib2.Http('.cache')
>>>response,content=
h.request('http://diveintopython3.org/examples/feed.xml')
>>>response.status
200
>>>content[:52]
b"<?xmlversion='1.0'encoding='utf8'?>\r\n<feed
xmlns="
>>>len(content)
3070
1.
httplib2 Http Http
httplib2
2.
Http ,
request() URL
HTTPGET .(
HTTP,POST)
3.
request()
httplib2.Response HTTP
,status 200
4.
contentHTTP
bytes
httplib2.Http
URL
Http
request()
HTTPLIB2
httplib2
httplib2
ContentTypeHTTP,
HTTP HTTP
HTTP
((contentsniffing)
)
(XML),
(bytes)
xml.etree.ElementTree.parse()(
)XML
XML
XML
ContentTypeHTTP,charset

XML
ContentTypeHTTP
RFC3023(
),ContentTypeHTTP(media
type)application/xml,application/xmldtd,
application/xmlexternalparsedentity,
application/xmlapplication/atom+xml
application/rss+xmlapplication/rdf+xml,
1.
2.
3.
ContentTypeHTTP charset ,
XML encoding ,
UTF8
ContentTypeHTTP
(mediatype) text/xml,text/xmlexternalparsedentity,
text/AnythingAtAll+xml ,
XML encoding
1.
2.
ContentTypeHTTP charset ,
usascii
XMLHTML
(byzantinerulesforcontent
sniffing)[PDF],.
.
HTTPLIB2
httplib2.Http
?
>>>response2,content2=
>>>response2.status
200
>>>content2[:52]
b"<?xmlversion='1.0'encoding='utf8'?>\r\n<feed
xmlns="
>>>len(content2)
3070
2.
HTTP(status) 200
3.
1.
Python shell
#NOTcontinuedfrompreviousexample!
#Pleaseexitoutoftheinteractiveshell
#andlaunchanewone.
>>>importhttplib2
>>>httplib2.debuglevel=1
>>>len(content)
3070
>>>response.status
200
>>>response.fromcache
True
1.
httplib2 http.client.httplib2
httplib2.Http
URL
4.
5.
HTTP
6.
: httplib2
httplib2.Http
httplib2
2.
3.
httplib2
(httplib2.debuglevel),
httplib2.Http
, httplib2.Http
URL(:
200)feed
24 (CacheControl:
maxage=86400,24 )httplib2
.cache(Http
)
URL,httplib2
httplib2 HTTP .
response.fromcache.
F5 Ctrl+F5

,()
HTTP
h.request('http://diveintopython3.org/examples/feed.xml',
...headers={'cachecontrol':'nocache'})
connect:(diveintopython3.org,80)
useragent:Pythonhttplib2/$Rev:259$
acceptencoding:deflate,gzip
cachecontrol:nocache'
>>>response2.status
200
>>>response2.fromcache
False
>>>print(dict(response2.items()))
{'status':'200',
'contentlength':'3070',
'contentlocation':
'http://diveintopython3.org/examples/feed.xml',
'acceptranges':'bytes',
'expires':'Wed,03Jun200900:40:26GMT',
'vary':'AcceptEncoding',
'server':'Apache',
'lastmodified':'Sun,31May200922:51:11GMT',
'connection':'close',
'contentencoding':'gzip',
'etag':'"bfe255ef5c0"',
'cachecontrol':'maxage=86400',
'date':'Tue,02Jun200900:40:26GMT',
'contenttype':'application/xml'}
1.
httplib2 HTTP
(
), headers
nocache
2.
httplib2 httplib2
. nocache
3.
4.
HTTP
,httplib2
HTTP
HTTPLIB2
LASTMODIFIEDETAG
CacheControlExpires(freshness
indicators)
:
,httplib2
().
,?HTTP
LastModifiedEtag
(validators)
304
>>>importhttplib2
h.request('http://diveintopython3.org/')
send:b'GET/HTTP/1.1
useragent:Pythonhttplib2/$Rev:259$'
>>>print(dict(response.items()))
{'contentencoding':'gzip',
'contentlocation':'http://diveintopython3.org/',
'contenttype':'text/html',
'date':'Tue,02Jun200903:26:54GMT',
'etag':'"7f806d1a019fb97900"',
'lastmodified':'Tue,02Jun200902:51:48GMT',
'server':'Apache',
'status':'200',
'vary':'AcceptEncoding,UserAgent'}
>>>len(content)
6657
HTML
httplib2
2.
HTTP
ETagLastModified
3.
6657
,
1.
send:b'GET/HTTP/1.1
ifnonematch:"7f806d1a019fb97900"
ifmodifiedsince:Tue,02Jun200902:51:48GMT
reply:'HTTP/1.1304NotModified'
True
>>>response.status
200
>>>response.dict['status']
'304'
>>>len(content)
6657
1.
2.
3.
4.
5.
6.
7.
8.
Http (
)
httplib2 ETagvalidator IfNoneMatch
httplib2 LastModifiedvalidator IfModified

Since
(validators),
, 304
.
httplib2 304
304(
, httplib2 ),200(
, httplib2 )
response.status
response.dict ,
.
content
(
httplib2
)request(),httplib2
HTTP2LIB
HTTPhttplib2
send:b'GET/HTTP/1.1
>>>print(dict(response.items()))
{'contentencoding':'gzip',
'contentlocation':'http://diveintopython3.org/',
'contenttype':'text/html',
'date':'Tue,02Jun200903:26:54GMT',
'etag':'"7f806d1a019fb97900"',
'lastmodified':'Tue,02Jun200902:51:48GMT',
'server':'Apache',
'status':'304',
'vary':'AcceptEncoding,UserAgent'}
httplib2 AcceptEncoding
deflategzip
2.
gzip request()
httplib2 (body)
content ,
response['contentencoding'];.
1.
HTTPLIB2
HTTP:
,httplib2
>>>importhttplib2
h.request('http://diveintopython3.org/examples/feed
302.xml')
send:b'GET/examples/feed302.xmlHTTP/1.1
reply:'HTTP/1.1302Found'
URL
2.
3.
:302Found
Location URL.
4.
httplib2 Location
URL:
http://diveintopython3.org/examples/feed.xml
1.
httplib2
URL,
.httplib2 URL .
>>>response
{'status':'200',
'contentlocation':
'expires':'Thu,04Jun200902:21:41GMT',
'vary':'AcceptEncoding',
'server':'Apache',
'lastmodified':'Wed,03Jun200902:20:15GMT',
'contentencoding':'gzip',
'etag':'"bfe4cbbf5c0"',
'date':'Wed,03Jun200902:21:41GMT',
'contenttype':'application/xml'}
request() response URL

httplib2URL contentlocation
response
httplib2
3.
,.
4.
.()
1.
2.
response URL
URL URL httplib2
>>>response.previous
{'status':'302',
'contentlocation':
'http://diveintopython3.org/examples/feed302.xml',
'expires':'Thu,04Jun200902:21:41GMT',
'server':'Apache',
'location':
'date':'Wed,03Jun200902:21:41GMT',
'contenttype':'text/html;charset=iso88591'}
>>>type(response)
<class'httplib2.Response'>
>>>type(response.previous)
<class'httplib2.Response'>
>>>response.previous.previous
>>>
response.previous
httplib2
responseresponse.previoushttplib2.Response
2.
1.
3.
response.previous.previous
(: URL
URL URL!)
None.
URL ?
302.xml')
reply:'HTTP/1.1302Found'
>>>content2==content
True
URL,httplib2.Http(
)
2.
302 httplib2URL
3.
302 :
URL,
( CacheControl
)httplib2 302Found,
.
,
4.
request() feed
1.
httplib2 URL
httplib2 ,ETags,
HTTP
301.xml')
reply:'HTTP/1.1301MovedPermanently'
True
URL
http://diveintopython3.org/examples/feed.xml.
2.
:301:
URL ?
3.
httplib2
1.
!!
301.xml')
>>>response2.fromcache
True
>>>content2==content
True
:httplib2
, URL
URL URL,,
2.
,
3.
,()
1.
HTTP.
HTTP GET
HTTPwebGET
Twitter
Identi.ca,
HTTPPOST.
TwitterIdenti.caHTTPAPI
( 140 )Identi.ca
API:
Identi.ca RESTAPI:statuses/update
status
POST.
URL
https://identi.ca/api/statuses/update.fo
rmat
Formats
xml,json,rss,atom
HTTPMethod(s)
POST
RequiresAuthentication
true
Parameters
status.Required.Thetextofyourstatus
update.URLencodeasnecessary.
Identi.ca,
HTTPPOST
http://identi.ca/api/statuses/update.format.(format
URL ;
XML
https://identi.ca/api/statuses/update.xml )
status,
?Identi.ca,
Identi.ca;
Identi.caSSLHTTPBasicAuthentication(
RFC2617)httplib2SSL
HTTPBasicAuthentication,
POSTGET,(payload).
APIstatus,
URL
()
>>>fromurllib.parseimporturlencode
>>>data={'status':'TestupdatefromPython3'}
>>>urlencode(data)
'status=Test+update+from+Python+3'
Python URL :
urllib.parse.urlencode().
2.
Identi.caAPIstatus,
3.
URL
Identi.caAPI HTTPPOST.
1.
>>>fromurllib.parseimporturlencode
>>>importhttplib2
>>>data={'status':'TestupdatefromPython3'}
>>>h.add_credentials('diveintomark',
'MY_SECRET_PASSWORD','identi.ca')
>>>resp,content=
h.request('https://identi.ca/api/statuses/update.xml',
...'POST',
...urlencode(data),
...headers={'ContentType':'application/xwww
formurlencoded'})
httplib2 add_credentials()
httplib2
401Unauthorized ,
(WWWAuthenticate).httplib2
Authorization URL.
2.
HTTP POST.
3.
URL
4.
URL
1.
add_credentials()
!
httplib2.Http
httplib2
:
send:b'POST/api/statuses/update.xmlHTTP/1.1
Host:identi.ca
ContentLength:32
contenttype:application/xwwwformurlencoded
status=Test+update+from+Python+3'
reply:'HTTP/1.1401Unauthorized'
send:b'POST/api/statuses/update.xmlHTTP/1.1
Host:identi.ca
ContentLength:32
contenttype:application/xwwwformurlencoded
authorization:BasicSECRET_HASH_CONSTRUCTED_BY_HTTPLIB2
status=Test+update+from+Python+3'
401Unauthorized
httplib2
2.
httplib2 URL
3.
add_credentials()
4.
!
1.
webAPI
(AtomPublishingProtocol),
201CreatedLocation
Identi.ca 200OKXML
>>>print(content.decode('utf8'))
<?xmlversion="1.0"encoding="UTF8"?>
<status>
<text>TestupdatefromPython3</text>
<truncated>false</truncated>
<created_at>WedJun1003:53:46+00002009</created_at>
<in_reply_to_status_id></in_reply_to_status_id>
<source>api</source>
<id>5131472</id>
<in_reply_to_user_id></in_reply_to_user_id>
<in_reply_to_screen_name></in_reply_to_screen_name>
<favorited>false</favorited>
<user>
<id>3212</id>
<name>MarkPilgrim</name>
<screen_name>diveintomark</screen_name>
<location>27502,US</location>
<description>techwriter,husband,
father</description>
<profile_image_url>http://avatar.identi.ca/321248
20081216000626.png</profile_image_url>
<url>http://diveintomark.org/</url>
<protected>false</protected>
<followers_count>329</followers_count>
<profile_background_color></profile_background_color>
<profile_text_color></profile_text_color>
<profile_link_color></profile_link_color>
<profile_sidebar_fill_color></profile_sidebar_fill_color
>
<profile_sidebar_border_color></profile_sidebar_border_c
olor>
<friends_count>2</friends_count>
<created_at>WedJul0222:03:58+0000
2008</created_at>
<favourites_count>30768</favourites_count>
<utc_offset>0</utc_offset>
<time_zone>UTC</time_zone>
<profile_background_image_url></profile_background_image
_url>
<profile_background_tile>false</profile_background_tile>
<statuses_count>122</statuses_count>
<following>false</following>
<notifications>false</notifications>
</user>
</status>
,httplib2 (bytes),
Identi.caAPIUTF8 ,
2.
3.
Identi.ca
web URL
1.
HTTP POST
HTTP GETPOST
web web API GET
POST , httplib2
>>>fromxml.etreeimportElementTreeasetree
>>>tree=etree.fromstring(content)
>>>status_id=tree.findtext('id')
>>>status_id
'5131472'
>>>url=
'https://identi.ca/api/statuses/destroy/{0}.xml'.format(
status_id)
>>>resp,deleted_content=h.request(url,'DELETE')
XML,?XML.
findtext()
<id>.
3.
<id> URL
4.
URL HTTPDELETE
1.
2.
send:b'DELETE/api/statuses/destroy/5131472.xml
HTTP/1.1
Host:identi.ca
'
reply:'HTTP/1.1401Unauthorized'
send:b'DELETE/api/statuses/destroy/5131472.xml
HTTP/1.1
Host:identi.ca
authorization:BasicSECRET_HASH_CONSTRUCTED_BY_HTTPLIB2
'
>>>resp.status
200
1.
2.
.
Dave,
3.
4.
5.
httplib2:
httplib2
httplib2
HTTP:httplib2
httplib2:HTTP
HTTP:
HTTPMarkNottingham
HTTPGoogleDoctype
RFCs:
RFC2616:HTTP
RFC2617:HTTPBasicAuthentication
RFC1951:deflatecompression
RFC1952:gzipcompression
20019MarkPilgrim
Search
:
:CHARDET
PYTHON 3
Words,words.Theyreallwehavetogoon.
RosencrantzandGuildensternareDead
(gibberish
text)
Unicode
(authoringsystem)
Unicode
pony
Unicode pony
Unipony
txzqJv
2!dasd0aQqdKjvz
(typical)
MozillaFirefox
Python2chardet
chardetPython
2 Python3
CHARDET
chardet
chardet
chardet.feedparser.org
universaldetector.py
UniversalDetector
chardet/__init__.py detect
UniversalDetector
UniversalDetector
1.
2.
3.
4.
5.
(BOM) UTFn UTF8

UTF16 UTF32
ASCII ASCII
ISO2022JP() HZ
GB2312().
Big5()SHIFT_JIS()EUCKR()
BOM UTF8
KOI8R()windows1255() TIS
620()
windows1252
(middlemanager) MicrosoftWindows
BOMUTF-N
BOM
UTF8UTF16 UTF32 BOM
UniversalDetector
UniversalDetector EscCharSetProber
escprober.py
EscCharSetProber HZGB2312ISO2022CNISO
2022JP ISO2022KR( escsm.py )
(statemachine)EscCharSetProber
EscCharSetProber
UniversalDetector UniversalDetector
BOM UniversalDetector
(highbitcharacter)
(probers)
windows1252
MBCSGroupProber
mbcsgroupprober.py
shellBig5GB2312
EUCTWEUCKREUCJPSHIFT_JIS UTF8
MBCSGroupProber
UniversalDetector.feed()
MBCSGroupProber
MultiByteCharSetProber(
mbcharsetprober.py )
(distributionanalyzer)
MultiByteCharSetProber
chardistribution.py
(distributionratio)(confidencerating)
MBCSGroupProber MBCSGroupProber
EUCJP SHIFT_JIS SJISProber

sjisprober.py
SJISContextAnalysis EUCJPContextAnalysis(
jpcntx.py JapaneseContextAnalysis
(Hiraganasyllabarycharacher)
SJISProberSJISProber
MBCSGroupProber
Unicodepony
SBCSGroupProber
sbcsgroupprober.py
shellwindows
1251KOI8RISO88595MacCyrillicIBM855and
IBM866()ISO88597 windows1253()ISO
88595 windows1251()ISO88592 windows
1250()TIS620()windows1255 ISO8859
8()
SBCSGroupProber +
SingleByteCharSetProber( sbcharsetprober.py )
(languagemodel)
SingleByteCharSetProber
HebrewProber(
hebrewprober.py ) VisualHebrew
LogicalHebrew
(windows1255 Logical
Hebrew ISO88598 VisualHebrew)
WINDOWS1252
UniversalDetector
Latin1Prober(
latin1prober.py)windows1252
windows1252
(smartquotes)(curlyapostrophes)
(copyrightsymbol)Latin1Prober
2TO3
chardetPython3 Python3
2to3 Python2
Python3
2to3 Python3
2to3chardet
chardet
2to3
2to3
C:\home\chardet>python
c:\Python30\Tools\Scripts\2to3.pywchardet\
RefactoringTool:Skippingimplicitfixer:buffer
RefactoringTool:Skippingimplicitfixer:idioms
RefactoringTool:Skippingimplicitfixer:set_literal
RefactoringTool:Skippingimplicitfixer:ws_comma
chardet\__init__.py(original)
+++chardet\__init__.py(refactored)
@@18,7+18,7@@
__version__="1.0.1"
defdetect(aBuf):
+from.importuniversaldetector
u=universaldetector.UniversalDetector()
u.reset()
u.feed(aBuf)
chardet\big5prober.py(original)
+++chardet\big5prober.py(refactored)
@@25,10+25,10@@
#021101301USA
#########################ENDLICENSEBLOCK
#########################
+from.mbcharsetproberimportMultiByteCharSetProber
+from.codingstatemachineimportCodingStateMachine
+from.chardistributionimportBig5DistributionAnalysis
+from.mbcssmimportBig5SMModel
classBig5Prober(MultiByteCharSetProber):
def__init__(self):
chardet\chardistribution.py(original)
+++chardet\chardistribution.py(refactored)
@@25,12+25,12@@
#021101301USA
#########################ENDLICENSEBLOCK
#########################
+from.importconstants
+from.euctwfreqimportEUCTWCharToFreqOrder,
EUCTW_TABLE_SIZE,EUCTW_TYPICAL_DISTRIBUTION_RATIO
+from.euckrfreqimportEUCKRCharToFreqOrder,
EUCKR_TABLE_SIZE,EUCKR_TYPICAL_DISTRIBUTION_RATIO
+from.gb2312freqimportGB2312CharToFreqOrder,
GB2312_TABLE_SIZE,GB2312_TYPICAL_DISTRIBUTION_RATIO
+from.big5freqimportBig5CharToFreqOrder,
BIG5_TABLE_SIZE,BIG5_TYPICAL_DISTRIBUTION_RATIO
+from.jisfreqimportJISCharToFreqOrder,JIS_TABLE_SIZE,
JIS_TYPICAL_DISTRIBUTION_RATIO
ENOUGH_DATA_THRESHOLD=1024
SURE_YES=0.99
.
.
.(itgoesonlikethisforawhile)
.
.
RefactoringTool:Filesthatweremodified:
RefactoringTool:chardet\__init__.py
RefactoringTool:chardet\big5prober.py
RefactoringTool:chardet\chardistribution.py
RefactoringTool:chardet\charsetgroupprober.py
RefactoringTool:chardet\codingstatemachine.py
RefactoringTool:chardet\constants.py
RefactoringTool:chardet\escprober.py
RefactoringTool:chardet\escsm.py
RefactoringTool:chardet\eucjpprober.py
RefactoringTool:chardet\euckrprober.py
RefactoringTool:chardet\euctwprober.py
RefactoringTool:chardet\gb2312prober.py
RefactoringTool:chardet\hebrewprober.py
RefactoringTool:chardet\jpcntx.py
RefactoringTool:chardet\langbulgarianmodel.py
RefactoringTool:chardet\langcyrillicmodel.py
RefactoringTool:chardet\langgreekmodel.py
RefactoringTool:chardet\langhebrewmodel.py
RefactoringTool:chardet\langhungarianmodel.py
RefactoringTool:chardet\langthaimodel.py
RefactoringTool:chardet\latin1prober.py
RefactoringTool:chardet\mbcharsetprober.py
RefactoringTool:chardet\mbcsgroupprober.py
RefactoringTool:chardet\mbcssm.py
RefactoringTool:chardet\sbcharsetprober.py
RefactoringTool:chardet\sbcsgroupprober.py
RefactoringTool:chardet\sjisprober.py
RefactoringTool:chardet\universaldetector.py
RefactoringTool:chardet\utf8prober.py
test.py 2to3
C:\home\chardet>python
c:\Python30\Tools\Scripts\2to3.pywtest.py
RefactoringTool:Skippingimplicitfixer:buffer
RefactoringTool:Skippingimplicitfixer:idioms
RefactoringTool:Skippingimplicitfixer:set_literal
RefactoringTool:Skippingimplicitfixer:ws_comma
test.py(original)
+++test.py(refactored)
@@4,7+4,7@@
count=0
u=UniversalDetector()
forfinglob.glob(sys.argv[1]):
+print(f.ljust(60),end='')
u.reset()
forlineinfile(f,'rb'):
u.feed(line)
@@12,8+12,8@@
u.close()
result=u.result
ifresult['encoding']:

+print(result['encoding'],'withconfidence',
result['confidence'])
else:
+print('********noresult')
count+=1
+print(count,'tests')
RefactoringTool:Filesthatweremodified:
RefactoringTool:test.py
impor print
import
chardet
chardet
( chardet.py)
( chardet)
__init__.py Python __init__.py
Python
.py
__init__.py
__init__.py
__init__.py
__init__.py .py
>>>importchardet
>>>dir(chardet)
['__builtins__','__doc__','__file__','__name__',
'__package__','__path__','__version__','detect']
>>>chardet
<module'chardet'from'C:\Python31\lib\site
packages\chardet\__init__.py'>
chardet detect()
2.
chardet
module chardet/__init__.py
1.
__init__.py
defdetect(aBuf):
from.importuniversaldetector
u=universaldetector.UniversalDetector()
u.reset()
u.feed(aBuf)
u.close()
returnu.result
1.
__init__.py detect() chardet
2.
detect()
universaldetector
universaldetector
import
from.importuniversaldetector
universaldetector
chardet/__init__.py
import
chardet/universaldetector
__init__.py
chardet 36 .py
36 chardet/__init__.py
detect()
chardet/universaldetector.py
5
chardet/
Python
Python
2TO3
FALSE IS INVALID SYNTAX
bug
C:\home\chardet>pythontest.pytests\*\*
File"test.py",line1,in<module>
fromchardet.universaldetectorimport
UniversalDetector
File"C:\home\chardet\chardet\universaldetector.py",
line51
self.done=constants.False
^
SyntaxError:invalidsyntax
Python3 False
constants.py
constants.py 2to3
import__builtin__
ifnothasattr(__builtin__,'False'):
False=0
True=1
else:
False=__builtin__.False
True=__builtin__.True
Python2 Python
2.3 Python bool
True False
Python3 bool
constants.True
constants.False True False
constants.py
self.done=constants.False
self.done=False
NO MODULE NAMED CONSTANTS

test.py
fromchardet.universaldetectorimport
UniversalDetector
line29,in<module>
importconstants,sys
ImportError:Nomodulenamedconstants
constants
constants chardet/constants.py
2to3
Python3 Python2
importconstants
chardet/Python3
Python3
from.importconstants
2to3
constants sys
sys Python
Python2 Python
3 2to3
importconstants,sys
importsys
chardet
importconstants,sysimport
constants,re
NAME 'FILE' IS NOT DEFINED

open() file()PapayaWhip black
test.py
tests\ascii\howto.diveintomark.org.xml
forlineinfile(f,'rb'):
NameError:name'file'isnotdefined

Python2 file()open()
open()
Python3 file()open()
file()
open()
forlineinopen(f,'rb'):
CANT USE A STRING PATTERN ON A BYTES-LIKE

OBJECT
u.feed(line)
line98,infeed
ifself._highBitDetector.search(aBuf):
TypeError:can'tuseastringpatternonabyteslike
object
self._highBitDetector
UniversalDetector __init__
classUniversalDetector:
def__init__(self):
self._highBitDetector=re.compile(r'[\x80
\xFF]')
128255
(0x800xFF) ASCII
128255
ASCII bytes
Python2
Python2
Unicode (u'') Python3
Python2 Unicode Unicode
traceback
deffeed(self,aBuf):
.
.
.
ifself._mInputState==ePureAscii:
aBuf UniversalDetector.feed()
test.py
u=UniversalDetector()
.
.
.
forlineinopen(f,'rb'):
u.feed(line)
aBuf
'rb''r'OK
'b'
'b'for
Unicode
'b'for
self._highBitDetector
def__init__(self):
+self._highBitDetector=re.compile(b'[\x80
\xFF]')
+self._escDetector=re.compile(b'(\033|~{)')
self._mEscCharSetProber=None
self._mCharSetProbers=[]
self.reset()
re
charsetprober.py
aBuf
aBuf
classCharSetProber:
.
.
.
deffilter_high_bit_only(self,aBuf):
+aBuf=re.sub(b'([\x00\x7F])+',b'',aBuf)
returnaBuf
deffilter_without_english_letters(self,aBuf):
+aBuf=re.sub(b'([AZaz])+',b'',aBuf)
returnaBuf
CAN'T CONVERT 'BYTES' OBJECT TO STR

IMPLICITLY
u.feed(line)
line100,infeed
elif(self._mInputState==ePureAscii)and
self._escDetector.search(self._mLastChar+aBuf):
implicitly
Python
TypeError traceback
(conditional) traceback
elif(self._mInputState==ePureAscii)and\
u.feed(line)
line101,infeed
implicitly
(self._mInputState==
ePureAscii)
TypeError search()
traceback
Python
traceback
traceback
search()
+ search()
aBuf
self._mLastCharreset()
reset()__init__()
def__init__(self):
self._highBitDetector=re.compile(b'[\x80
\xFF]')
self._escDetector=re.compile(b'(\033|~{)')
self._mEscCharSetProber=None
self._mCharSetProbers=[]
self.reset()
defreset(self):
self.result={'encoding':None,'confidence':
0.0}
self.done=False
self._mStart=True
self._mGotData=False
self._mInputState=ePureAscii
self._mLastChar=''
self._mLastChar aBuf
self._mLastChar feed()
traceback
self._mInputState=eHighbyte
self._escDetector.search(self._mLastChar+
aBuf):
self._mInputState=eEscAscii
self._mLastChar=aBuf[1]
feed()
aBuf
self._mLastChar
feed()
aBuf self._mLastChar
defreset(self):
.
.
.
+self._mLastChar=b''
mLastCharmbcharsetprober.py
2
Python3
0255
classMultiByteCharSetProber(CharSetProber):
def__init__(self):
CharSetProber.__init__(self)
self._mDistributionAnalyzer=None
self._mCodingSM=None

+self._mLastChar=[0,0]
defreset(self):
CharSetProber.reset(self)
ifself._mCodingSM:
self._mCodingSM.reset()
ifself._mDistributionAnalyzer:
self._mDistributionAnalyzer.reset()
+self._mLastChar=[0,0]
UNSUPPORTED OPERAND TYPE(S) FOR +: 'INT'

AND 'BYTES'
u.feed(line)
line101,infeed
TypeError:unsupportedoperandtype(s)for+:'int'and
'bytes'

traceback
int(bytes)
self._mLastChar
int
self._mInputState=eHighbyte
aBuf):
self._mLastChar=aBuf[1]
feed()
self._mLastChar aBuf
shell
>>>aBuf=b'\xEF\xBB\xBF'
>>>len(aBuf)
3
>>>mLastChar=aBuf[1]
>>>mLastChar
191
>>>type(mLastChar)
<class'int'>
>>>mLastChar+aBuf
TypeError:unsupportedoperandtype(s)for+:'int'and
'bytes'
>>>mLastChar=aBuf[1:]
>>>mLastChar
b'\xbf'
>>>mLastChar+aBuf
b'\xbf\xef\xbb\xbf'
3
191
5.
mLastChar 1
1.
2.
3.
4.
6.
1 3
4
universaldetector.pyfeed()
self._mLastChar
0
aBuf):
+self._mLastChar=aBuf[1:]
ORD() EXPECTED STRING OF LENGTH 1, BUT INT

FOUND
asciiwithconfidence1.0
tests\Big5\0804.blogspot.com.xml
u.feed(line)
line116,infeed
ifprober.feed(aBuf)==constants.eFoundIt:
File"C:\home\chardet\chardet\charsetgroupprober.py",
line60,infeed
st=prober.feed(aBuf)
File"C:\home\chardet\chardet\utf8prober.py",line53,
infeed
codingState=self._mCodingSM.next_state(c)
File"C:\home\chardet\chardet\codingstatemachine.py",
line43,innext_state
byteCls=self._mModel['classTable'][ord(c)]
TypeError:ord()expectedstringoflength1,butint
found
OK c int ord() 1
c
#codingstatemachine.py
defnext_state(self,c):
#foreachbytewegetitsclass
#ifitisfirstbyte,wealsogetbytelength
byteCls=self._mModel['classTable'][ord(c)]
c next_state()
#utf8prober.py
deffeed(self,aBuf):
forcinaBuf:
codingState=self._mCodingSM.next_state(c)
Python2 aBuf c
1
aBuf
c int 1
ord() c int
defnext_state(self,c):
#foreachbytewegetitsclass
#ifitisfirstbyte,wealsogetbytelength
+byteCls=self._mModel['classTable'][c]
ord(c) sbcharsetprober.py
#sbcharsetprober.py
deffeed(self,aBuf):
ifnotself._mModel['keepEnglishLetter']:
aBuf=self.filter_without_english_letters(aBuf)
aLen=len(aBuf)
ifnotaLen:
returnself.get_state()
forcinaBuf:
order=self._mModel['charToOrderMap'][ord(c)]
latin1prober.py
#latin1prober.py
deffeed(self,aBuf):
aBuf=self.filter_with_english_letters(aBuf)
forcinaBuf:
charClass=Latin1_CharToClass[ord(c)]
c aBuf
ord(c) c
#sbcharsetprober.py
deffeed(self,aBuf):
ifnotself._mModel['keepEnglishLetter']:
aBuf=
self.filter_without_english_letters(aBuf)
aLen=len(aBuf)
ifnotaLen:
returnself.get_state()
forcinaBuf:
+order=self._mModel['charToOrderMap'][c]
#latin1prober.py
deffeed(self,aBuf):
aBuf=self.filter_with_english_letters(aBuf)
forcinaBuf:

+charClass=Latin1_CharToClass[c]
UNORDERABLE TYPES: INT() >= STR()
u.feed(line)
line116,infeed
ifprober.feed(aBuf)==constants.eFoundIt:
File"C:\home\chardet\chardet\charsetgroupprober.py",
line60,infeed
st=prober.feed(aBuf)
File"C:\home\chardet\chardet\sjisprober.py",line68,
infeed
self._mContextAnalyzer.feed(self._mLastChar[2
charLen:],charLen)
File"C:\home\chardet\chardet\jpcntx.py",line145,in
feed
order,charLen=self.get_order(aBuf[i:i+2])
File"C:\home\chardet\chardet\jpcntx.py",line176,in
get_order
if((aStr[0]>='\x81')and(aStr[0]<='\x9F'))or
\
TypeError:unorderabletypes:int()>=str()
Unorderabletypes
classSJISContextAnalysis(JapaneseContextAnalysis):
defget_order(self,aStr):
ifnotaStr:return1,1
#findoutcurrentchar'sbytelength
if((aStr[0]>='\x81')and(aStr[0]<='\x9F'))
or\
((aStr[0]>='\xE0')and(aStr[0]<='\xFC')):
charLen=2
else:
charLen=1
aStr
deffeed(self,aBuf,aLen):
.
.
.
i=self._mNeedToSkipCharNum
whilei<aLen:
order,charLen=self.get_order(aBuf[i:i+2])
aBuf
aBuf
feed()
get_order()aStr
aStr
1 Python2
aStr aBuf aStr[0]
Python3
aStr aBuf aStr[0]
aStr[0]
(constant) 1
aStr aBuf aStr
classSJISContextAnalysis(JapaneseContextAnalysis):
+defget_order(self,aBuf):
+ifnotaBuf:return1,1
+if((aBuf[0]>=0x81)and(aBuf[0]<=0x9F))
or\
+((aBuf[0]>=0xE0)and(aBuf[0]<=0xFC)):
charLen=2
else:
charLen=1
#returnitsorderifitishiragana
+iflen(aBuf)>1:
+if(aBuf[0]==0x202)and\
+(aBuf[1]>=0x9F)and\
+(aBuf[1]<=0xF1):
+returnaBuf[1]0x9F,charLen
return1,charLen
classEUCJPContextAnalysis(JapaneseContextAnalysis):
+defget_order(self,aBuf):
+ifnotaBuf:return1,1
+if(aBuf[0]==0x8E)or\
+((aBuf[0]>=0xA1)and(aBuf[0]<=0xFE)):
charLen=2
+elifaBuf[0]==0x8F:
charLen=3
else:
charLen=1
#returnitsorderifitishiragana
+iflen(aBuf)>1:
+if(aBuf[0]==0xA4)and\
+(aBuf[1]>=0xA1)and\
+(aBuf[1]<=0xF3):
+returnaBuf[1]0xA1,charLen
return1,charLen
ord() chardistribution.py
EUCTWDistributionAnalysisEUCKRDistributionAnalysis
GB2312DistributionAnalysisBig5DistributionAnalysis
SJISDistributionAnalysis EUCJPDistributionAnalysis
jpcntx.py
EUCJPContextAnalysis SJISContextAnalysis
GLOBAL NAME 'REDUCE' IS NOT DEFINED
u.close()
line141,inclose
proberConfidence=prober.get_confidence()
File"C:\home\chardet\chardet\latin1prober.py",line
126,inget_confidence
total=reduce(operator.add,self._mFreqCounter)
NameError:globalname'reduce'isnotdefined
WhatsNewInPython3.0reduce()
functools
functools.reduce()99
forGuidovan
RossumThefate
ofreduce()inPython3000
defget_confidence(self):
ifself.get_state()==constants.eNotMe:
return0.01
total=reduce(operator.add,self._mFreqCounter)
reduce()
(addup)
Python
sum()
defget_confidence(self):
ifself.get_state()==constants.eNotMe:
return0.01
+total=sum(self._mFreqCounter)
operator
import
from.charsetproberimportCharSetProber
Big5withconfidence0.99
tests\Big5\blog.worren.net.xml
tests\Big5\carbonxiv.blogspot.com.xml
tests\Big5\catshadow.blogspot.com.xml
tests\Big5\coolloud.org.tw.xml
tests\Big5\digitalwall.com.xml
tests\Big5\ebao.us.xml
tests\Big5\fudesign.blogspot.com.xml
tests\Big5\kafkatseng.blogspot.com.xml
tests\Big5\ke207.blogspot.com.xml
tests\Big5\leavesth.blogspot.com.xml
tests\Big5\letterlego.blogspot.com.xml
tests\Big5\linyijen.blogspot.com.xml
tests\Big5\marilynwu.blogspot.com.xml
tests\Big5\myblog.pchome.com.tw.xml
tests\Big5\ouidesign.com.xml
tests\Big5\sanwenji.blogspot.com.xml
tests\Big5\sinica.edu.tw.xml
tests\Big5\sylvia1976.blogspot.com.xml
tests\Big5\tlkkuo.blogspot.com.xml
tests\Big5\tw.blog.xubg.com.xml
tests\Big5\unoriginalblog.com.xml
tests\Big5\upsaid.com.xml
tests\Big5\willythecop.blogspot.com.xml
tests\Big5\ytc.blogspot.com.xml
tests\EUCJP\aivy.co.jp.xml
EUCJPwithconfidence0.99
tests\EUCJP\akaname.main.jp.xml
tests\EUCJP\arclamp.jp.xml
.
.
.
316tests
/medoesalittledance
1.
2.
3.
4.
5.
Python2 Python3
2to3
chardet
chardet
web
webAPI
bug
chardet Python3
20019MarkPilgrim
:Python3
PYTHON
Youllfindtheshameislikethepain;youonlyfeelitonce.
MarquisedeMerteuil,DangerousLiaisons
Python
Python
Python3DistutilsDistutils
Python
PyPIPython
Distutilssetupscript
setup.pyDistutils
HTTPWebServicesDistutils
httplib2chardetPython3
chardet
chardethttplib2
Python
#chardet'ssetup.py
fromdistutils.coreimportsetup
setup(
name="chardet",
packages=["chardet"],
version="1.0.2",
description="Universalencodingdetector",
author="MarkPilgrim",
author_email="mark@diveintomark.org",
url="http://chardet.feedparser.org/",
download_url=
"http://chardet.feedparser.org/download/python3chardet
1.0.1.tgz",
keywords=["encoding","i18n","xml"],
classifiers=[
"ProgrammingLanguage::Python",
"ProgrammingLanguage::Python::3",
"DevelopmentStatus::4Beta",
"Environment::OtherEnvironment",
"IntendedAudience::Developers",
"License::OSIApproved::GNULibraryor
LesserGeneralPublicLicense(LGPL)",
"OperatingSystem::OSIndependent",
"Topic::SoftwareDevelopment::Libraries::
PythonModules",
"Topic::TextProcessing::Linguistic",
],
long_description="""\
Universalcharacterencodingdetector
Detects
ASCII,UTF8,UTF16(2variants),UTF32(4variants)
Big5,GB2312,EUCTW,HZGB2312,ISO2022CN
(TraditionalandSimplifiedChinese)
EUCJP,SHIFT_JIS,ISO2022JP(Japanese)
EUCKR,ISO2022KR(Korean)
KOI8R,MacCyrillic,IBM855,IBM866,ISO88595,
windows1251(Cyrillic)
ISO88592,windows1250(Hungarian)
ISO88595,windows1251(Bulgarian)
windows1252(English)
ISO88597,windows1253(Greek)
ISO88598,windows1255(VisualandLogicalHebrew)
TIS620(Thai)
ThisversionrequiresPython3orlater;aPython2
versionisavailableseparately.
"""
)
chardethttplib2
Python
Python
DISTUTILS
Python
Distutils
1.
2.
3.
4.
5.
GPLGPL
PyPI
(readme)
Python
httplib2
httplib2/
|
+README.txt
|
+setup.py
|
+httplib2/
|
+__init__.py
|
+iri2uri.py
Python
2.
Windows"".txt
Windows
LinuxMacOSX
Windows
3.
Distutilssetup.py
4.
Python.py
""httplib2
.py
httplib2httplib2/
__init__.pyhttplib2/
1.
chardethttplib2
chardet/chardet/
README.txtdocs/chardetHTML
docs/.html.css
images/.png.gif
(L)GPL
COPYING.txtLGPL
chardet/
|
+COPYING.txt
|
+setup.py
|
+README.txt
|
+docs/
||
|+index.html
||
|+usage.html
||
|+images/...
|
+chardet/
|
+__init__.py
|
+big5freq.py
|
+...
DistutilsPython
Python
Distutils
setup()Distutils95
Distutilssetup()
Distutils
setup()
setup()
setup()
name
version
author
author_email
url
PyPI
description
long_descriptionreStructuredTextformat
PyPIHTML
classifiers
PEP314
chardet
packages
setup(
name='chardet',
packages=['chardet'],
version='1.0.2',
description='Universalencodingdetector',
author='MarkPilgrim',
...
)
packages
Pythonpackages
chardet
...packagesDistutilschardet/
__init__.pychardet
.py
PythonPyPIPython
PyPI
classifiersDistutilssetup()
classifers
PyPI
Distutils
<b0 ."ProgrammingLanguage::
Python""ProgrammingLanguage::Python::3"
Python3
pypi.python.org
.
0. Windows MacOS
X Linux
"Operating
System::OSIndependent"
. beta Alpha
Prealpha
.
DevelopersEndUsers/DesktopScience/Research
SystemAdministrators
.DjangoZope
Framework
.
DjangoWeb
BSDWeb
DjangoPython3
ProgrammingLanguage::Python::3
ProgrammingLanguage::Python
License::OSIApproved::BSDLicense
OperatingSystem::OSIndependent
DevelopmentStatus::5Production/Stable
Environment::WebEnvironment
Framework::Django
IntendedAudience::Developers
Topic::Internet::WWW/HTTP
Topic::Internet::WWW/HTTP::DynamicContent
Topic::Internet::WWW/HTTP::WSGI
Topic::SoftwareDevelopment::Libraries::Python
Modules
chardetchardet
Python3chardet
Python3LGPL
License::OSIApproved::GNULibraryorLesserGeneral
PublicLicense(LGPL)
DevelopmentStatus::4Beta
Environment::OtherEnvironment
Topic::TextProcessing::Linguistic
Modules
httplib2HTTP
httplib2MIT
Python
License::OSIApproved::MITLicense
DevelopmentStatus::4Beta
Environment::WebEnvironment
Topic::Internet::WWW/HTTP
Modules
Distutils
README.txt
setup.py
packages.py
py_modules.py
httplib2chardet
COPYING.txtHTML
docs/Distutilschardet
manifestfile
MANIFEST.in
README.txtsetup.py
PythonDistutils
chardet
includeCOPYING.txt
recursiveincludedocs*.html*.css*.png*.gif
COPYING.txt
2.
recursiveinclude
docs/
.html.css.png.gif
1.
recursiveinclude.html.png
docs/
chardetXML
HTML XML
HTML)
Distutils
I
Distutils
Distutils
versionDistutils
c:\Users\pilgrim\chardet>c:\python31\python.exe
setup.pycheck
runningcheck
warning:check:missingrequiredmetadata:version
version
check
setup.pycheck
runningcheck
Distutils
Distutilsread
me
sdistDistutils
setup.pysdist
runningsdist
runningcheck
readingmanifesttemplate'MANIFEST.in'
writingmanifestfile'MANIFEST'
creatingchardet1.0.2
creatingchardet1.0.2\chardet
creatingchardet1.0.2\docs
creatingchardet1.0.2\docs\images
copyingfilestochardet1.0.2...
copyingCOPYING>chardet1.0.2
copyingREADME.txt>chardet1.0.2
copyingsetup.py>chardet1.0.2
copyingchardet\__init__.py>chardet1.0.2\chardet
copyingchardet\big5freq.py>chardet1.0.2\chardet
...
copyingchardet\universaldetector.py>chardet
1.0.2\chardet
copyingchardet\utf8prober.py>chardet1.0.2\chardet
copyingdocs\faq.html>chardet1.0.2\docs
copyingdocs\history.html>chardet1.0.2\docs
copyingdocs\howitworks.html>chardet1.0.2\docs
copyingdocs\index.html>chardet1.0.2\docs
copyingdocs\license.html>chardet1.0.2\docs
copyingdocs\supportedencodings.html>chardet
1.0.2\docs
copyingdocs\usage.html>chardet1.0.2\docs
copyingdocs\images\caution.png>chardet
1.0.2\docs\images
copyingdocs\images\important.png>chardet
1.0.2\docs\images
copyingdocs\images\note.png>chardet
1.0.2\docs\images
copyingdocs\images\permalink.gif>chardet
1.0.2\docs\images
copyingdocs\images\tip.png>chardet1.0.2\docs\images
copyingdocs\images\warning.png>chardet
1.0.2\docs\images
creatingdist
creating'dist\chardet1.0.2.zip'andadding'chardet
1.0.2'toit
adding'chardet1.0.2\COPYING'
adding'chardet1.0.2\PKGINFO'
adding'chardet1.0.2\README.txt'
adding'chardet1.0.2\setup.py'
adding'chardet1.0.2\chardet\big5freq.py'
adding'chardet1.0.2\chardet\big5prober.py'
...
adding'chardet1.0.2\chardet\universaldetector.py'
adding'chardet1.0.2\chardet\utf8prober.py'
adding'chardet1.0.2\chardet\__init__.py'
adding'chardet1.0.2\docs\faq.html'
adding'chardet1.0.2\docs\history.html'
adding'chardet1.0.2\docs\howitworks.html'
adding'chardet1.0.2\docs\index.html'
adding'chardet1.0.2\docs\license.html'
adding'chardet1.0.2\docs\supportedencodings.html'
adding'chardet1.0.2\docs\usage.html'
adding'chardet1.0.2\docs\images\caution.png'
adding'chardet1.0.2\docs\images\important.png'
adding'chardet1.0.2\docs\images\note.png'
adding'chardet1.0.2\docs\images\permalink.gif'
adding'chardet1.0.2\docs\images\tip.png'
adding'chardet1.0.2\docs\images\warning.png'
removing'chardet1.0.2'(andeverythingunderit)
Distutils (MANIFEST.in)
Distutils
COPYING.txtdocs/HTML
Distutils
dist/dist/.zip
c:\Users\pilgrim\chardet>dirdist
VolumeSerialNumberisDED5B4F8
Directoryofc:\Users\pilgrim\chardet\dist
07/30/200906:29PM<DIR>.
07/30/200906:29PM<DIR>..
07/30/200906:29PM206,440chardet1.0.2.zip
1File(s)206,440bytes
PythonWindows
Windows
Windows
bdist_wininstDistutils
Windows
setup.pybdist_wininst
runningbdist_wininst
runningbuild
runningbuild_py
creatingbuild
creatingbuild\lib
creatingbuild\lib\chardet
copyingchardet\big5freq.py>build\lib\chardet
copyingchardet\big5prober.py>build\lib\chardet
...
copyingchardet\universaldetector.py>
build\lib\chardet
copyingchardet\utf8prober.py>build\lib\chardet
copyingchardet\__init__.py>build\lib\chardet
installingtobuild\bdist.win32\wininst
runninginstall_lib
creatingbuild\bdist.win32
creatingbuild\bdist.win32\wininst
creatingbuild\bdist.win32\wininst\PURELIB
creatingbuild\bdist.win32\wininst\PURELIB\chardet
copyingbuild\lib\chardet\big5freq.py>
build\bdist.win32\wininst\PURELIB\chardet
copyingbuild\lib\chardet\big5prober.py>
...
copyingbuild\lib\chardet\universaldetector.py>
copyingbuild\lib\chardet\utf8prober.py>
copyingbuild\lib\chardet\__init__.py>
Writingbuild\bdist.win32\wininst\PURELIB\chardet1.0.2
py3.1.egginfo
creating
'c:\users\pilgrim\appdata\local\temp\tmp2f4h7e.zip'and
adding'.'toit
adding'PURELIB\chardet1.0.2py3.1.egginfo'
adding'PURELIB\chardet\big5freq.py'
adding'PURELIB\chardet\big5prober.py'
...
adding'PURELIB\chardet\universaldetector.py'
adding'PURELIB\chardet\utf8prober.py'
adding'PURELIB\chardet\__init__.py'
removing'build\bdist.win32\wininst'(andeverything
underit)
VolumeSerialNumberisAADEE29F
Directoryofc:\Users\pilgrim\chardet\dist
07/30/200910:14PM<DIR>.
07/30/200910:14PM<DIR>..
07/30/200910:14PM371,236chardet
1.0.2.win32.exe
07/30/200906:29PM206,440chardet1.0.2.zip
2File(s)577,676bytes
DistutilsLinux
Linux
Linux
chardetDebianGNU/Linux
Ubuntu
Debian
PythonDebianpythonchardet
Debian
Debian
Distutils Linux
PYTHON
Python
1.
2.
3.
setup.pysdistsetup.pybdist_*
PyPI
Register
PGPGPG
PyPI
PyPI
setup.pyregistersdistbdist_wininstupload
runningregister
Weneedtoknowwhoyouare,sopleasechooseeither:
1.useyourexistinglogin,
2.registerasanewuser,
3.havetheservergenerateanewpasswordforyou(and
emailittoyou),or
4.quit
Yourselection[default1]:1
Username:MarkPilgrim
Password:
Registeringchardettohttp://pypi.python.org/pypi
Serverresponse(200):OK
runningsdist
...outputtrimmedforbrevity...
runningbdist_wininst
...outputtrimmedforbrevity...
runningupload
Submittingdist\chardet1.0.2.zipto
http://pypi.python.org/pypi
Submittingdist\chardet1.0.2.win32.exeto
http://pypi.python.org/pypi
IcanstoreyourPyPIloginsofuturesubmissionswill
befaster.
(theloginwillbestoredinc:\home\.pypirc)
Saveyourlogin(y/N)?n
1.
2.
3.
4.
5.
6.
7.
8.
Distutils
Python URL
setup.py
(sdist)Windows
(bdist_wininst) PyPI(upload)
1ENTERuseyourexisting
login.
PyPIDistuils
DistutilsPython
Windows
Python
PyPI
Python
http://pypi.python.org/pypi/NAMENAME
setup.pyname
setup.py
:
setup.pyregistersdistbdist_wininstupload
PYTHON
Distutils Python
2009 8 Python3
Python2
Python3
Setuptools
Pip
Distribute
virtualenv
zc.buildout
Paver
Fabric
py2exe
Distutils
DistutilsPython
setup()
Distutils
PEP370:sitepackages
PEP370environmentstew
Python
Pip
Python
Python
20019MarkPilgrim
Search
:
2TO3
PYTHON 3
Lifeispleasant.Deathispeaceful.Itsthetransitionthats
troublesome.
IsaacAsimov(attributed)
Python2
Python3 Python3
2to3 (UtilityScript)
Python2 Python3
:chardetPython3(portingchardetto
Python3)
PRINT
Python2 print
printPython3 print()
print()
Notes
Python2
print
print1
Python3
print()
print(1)
print1,2
print(1,2)
print1,2,
print(1,2,
end='')
print(1,2,3,
file=sys.stderr)
print>>sys.stderr,
1,2,3
print()
print()
3.
print()
4.
Python2
(,) print
(trailingspace)
(carriagereturn) Python3 end=''
print() end
'\n' end
5.
Python2 >>pipe_name
sys.stderr Python3
file print()
file std.stdout
print()
1.
2.
UNICODE
Python2 UnicodeUnicode
Python3 Unicode(Unicodestrings)
Notes
Python2
Python3
u'PapayaWhip'
'PapayaWhip'
ur'PapayaWhip\foo'
r'PapayaWhip\foo'
Python2 Unicode Python3

Python3 Unicode
2.
Unicode (rawstring)(Python
"\") Python
3 Unicode
1.
UNICODE()
Python2
unicode()Unicodestr()
UnicodePython3 Unicode
str()(unicode()
Python3 )
Notes
Python2
Python3
unicode(anything)
str(anything)
LONG
Python2 intlongint
sys.maxint
Lint
Python3
intPython2
PEP237
Notes
Python2
Python3
x=1000000000000L
long(x)
x=
1000000000000
x=
0xFFFFFFFFFFFF
int(x)
type(x)islong
type(x)isint
isinstance(x,long)
isinstance(x,
int)
x=0xFFFFFFFFFFFFL
1.
2.
3.
4.
5.
Python2 Python3
Python2 Python3
Python3
long()
int()
int ( long)
isinstance()
int long
<>
Python2 <>!=Python3 !=
<>
Notes
Python
3
Python2
ifx<>y:
if
x!=
y:
if
x!=
y!=
z:
ifx<>y<>z:
1.
2.
HAS_KEY()
Python2 has_key()
(key)Python3
in
Notes
Python2
Python3
a_dictionary.has_key('PapayaWhip') 'PapayaWhip'
in
a_dictionary.has_key(x)or
a_dictionary.has_key(y)
a_dictionary.has_key(xory)
a_dictionary.has_key(x+y)
x+a_dictionary.has_key(y)
a_dictionary
xin
a_dictionary
oryin
a_dictionary
(xory)in
a_dictionary
(x+y)in
a_dictionary
x+(yin
a_dictionary)
or in
3.
orin
(Python
xoryx(x)
yPythona_dictionary)
4.
in +
2to3
in +
5.
1.
2.
Python2
keysitems values Python3
(dynamicview)
Python3
Python3
(choke)(view)(indexing)
Notes
Python2
Python3
a_dictionary.keys()
list(a_dictionary.keys())
a_dictionary.items()
list(a_dictionary.items())
a_dictionary.iterkeys()
iter(a_dictionary.keys())
[iforiin
[iforiin
a_dictionary.iterkeys()] a_dictionary.keys()]

1.
2.
3.
4.
5.
min(a_dictionary.keys()) nochange
list() keys()
2to3
( items())2to3
values()
Python3 iterkeys() iter()
keys()
2to3 iterkeys()
Python 3 keys()( iter()
)
2to3 keys()
keys()
min()
min()max()sum()list()tuple()set()
sorted()any() all()
Python2 Python3
HTTP
Python3 HTTP
http
Notes
Python2
Python3
importhttplib
import
http.client
import
http.cookies
import
http.cookiejar
importCookie
importcookielib
importBaseHTTPServer
import
http.server
import
SimpleHTTPServer
1.
importCGIHttpServer
http.client HTTP
HTTP
2.
http.cookies (Pythonic)
HTTP (HTTPheader)SetCookie cookies
3.
cookies
http.cookiejar
http.server HTTP
4.
URLLIB
Python2 URL
Python3
urllib
Notes
Python2
Python3
importurllib
fromurllibimport
import
urllib.request,
urllib.parse,
urllib.error
import
urllib.request,
urllib.error
import
urllib.parse
import
urllib.robotparser
from
FancyURLopener
urllib.request
fromurllibimport
import
urlencode
FancyURLopener
importurllib2
importurlparse
importrobotparser
fromurllib.parse
importurlencode
2.
3.
4.
5.
6.
from
Request
urllib.request
fromurllib2import
importRequest
HTTPError
fromurllib.error
importHTTPError
1.
fromurllib2import
Python2 urllib
urlopen() URL
splittype()splithost() splituser()
urllib 2to3
Python3 urllib2 urllib

urllib2
Python3 urllib build_opener()
Request HTTPBasicAuthHandler friends
Python3 urllib.parse Python2
urlparse
urllib.robotparserobots.txt
HTTP FancyURLopener
Python3 urllib.request urlencode()
urllib.parse
Request urllib.request
HTTPError urllib.error
2to3
Python2 urllib urllib.urlopen()
2to3 import
Notes Python2
importurllib
importurllib
print
print(urllib.
urllib.urlopen('http://diveintopython3.org/').read()
Python3
DBM
DBM (DBMclone) dbm
GNUDBM
dbm
Notes
Python2
Python3
importdbm
import
dbm.ndbm
import
dbm.gnu
import
dbm.bsd
import
dbm.dumb
import
dbm
importgdbm
importdbhash
importdumbdbm
importanydbm
importwhichdb
XMLRPC
XMLRPC HTTP RPC
XMLRPC XMLRPC
xmlrpc
Notes
Python2
Python3
importxmlrpclib
import
xmlrpc.client
import
xmlrpc.server
importDocXMLRPCServer
import
SimpleXMLRPCServer
Notes
Python2
Python3
try:
importio
importcStringIOas
StringIO
exceptImportError:
importStringIO
try:
import
pickle
importcPickleas
pickle
exceptImportError:
1.
2.
3.
4.
5.
importpickle
import__builtin__
importcopy_reg
importQueue
importSocketServer
importConfigParser
importrepr
importcommands
import
builtins
import
copyreg
importqueue
import
socketserver
import
configparser
import
reprlib
import
subprocess
Python2 cStringIO
StringIO StringIO
Python3 io
Python2 pickle
Python3 pickle
builtins Python
builtins
copyreg C
pickle
queue (multiproducer,multi
consumerqueue)
socketserver socketserver
6.
configparser INIstyle
reprlib repr()
7.
8.
9.
subprocess
Python2
import
foo fromfooimportBarPython2
foo.py Python (sys.path)
Python3 Python3
Python
chardet/
|
+__init__.py
|
+constants.py
|
+mbcharsetprober.py
|
+universaldetector.py
universaldetector.py constants.py
mbcharsetprober.py ?
Notes
Python2
Python3
importconstants
from.import
constants
frommbcharsetprober
from.mbcharsetprober
import
import
MultiByteCharSetProber MultiByteCharsetProber
from.
import (.)
(universaldetector.py)(constants.py)
(from..import
anothermodule)
2.
(slash)
mbcharsetprober.py universaldetector.py
(from..
importanothermodule)
1.
NEXT()
Python2 next()
Python3
next()
Notes Python2
Python3
anIterator.next()
next(anIterator)
a_function_that_returns_an_iterator().next() next(a_function_that_
classA:
classA:
defnext(self):
def__next__(self
pass
pass
classA:
nochange
defnext(self,x,y):
pass
next=42
next=42
foran_iteratorina_sequence_of_iterators: foran_iteratorina_
1.
2.
3.
4.
5.
an_iterator.next()
an_iterator.__nex
next()
next()
next()(2to3
)
Python3 __next__()
next()
2to3
next()
next
Python3 next()
__next__()
(
next 2to3 )
FILTER()
Python2 filter()
True False
Python3 filter()
Notes
Python2
Python3
filter(a_function,
a_sequence)
list(filter(a_function,
a_sequence))
filter(None,
a_sequence)
foriinfilter(None,
a_sequence):
[iforiin
filter(a_function,
a_sequence)]
list(filter(a_function,
a_sequence))
nochange
[iforiina_sequence
ifi]
nochange
nochange
1.
2.
3.
4.
5.
2to3 list()
filter()list()
filter() list()2to3
filter()
filter(None,...)2to3
for
filter() filter()
MAP()
filter()map()
(Python2 )
Notes
1.
2.
3.
4.
5.
Python2
Python3
map(a_function,
'PapayaWhip')
map(None,
'PapayaWhip')
map(lambdax:x+1,
range(42))
foriin
map(a_function,
a_sequence):
[iforiin
map(a_function,
a_sequence)]
list(map(a_function,
'PapayaWhip'))
list('PapayaWhip')
[x+1forxin
range(42)]
nochange
nochange
filter()2to3
list() map()
map(None,...) filter(None,...)
2to3 list()
map() lambda 2to3
for
map()
REDUCE()
Python3 reduce()
fucntools
Notes
Python2
Python3
reduce(a,b,c)
from
functools
import
reduce
reduce(a,
b,c)
APPLY()
Python2 apply() f
[a,b,c] f(a,b,c)
(*)
Python3 apply()
Notes Python2
apply(a_function,
a_list_of_args)
apply(a_function,
a_list_of_args,
a_dictionary_of_named_args)
apply(a_function,
a_list_of_args+z)
apply(aModule.a_function,
a_list_of_args)
Python3
a_function(*a_list_of_args)
a_function(*a_list_of_args,
**a_dictionary_of_named_args)
a_function(*a_list_of_args+z)
aModule.a_function(*a_list_of_args)
1.
([a,b,c])
Python2 apply()
2.
Python2 apply() 3
(dictionaryof
namedarguments) Python3
(*)(**)
3.
+
* a_list_of_args+z
4.
2to3 apply()
INTERN()
Python2 intern()
(intern) Python3
intern() sys
Notes
Python2
Python3
intern(aString)
sys.intern(aString)
EXEC
printPython3 exec
exec()Python
exec()
eval()exec()
eval()exec()
(import)Python
Notes
1.
Python2
Python3
execcodeString
exec(codeString)
execcodeStringin
a_global_namespace
execcodeStringin
a_global_namespace,
a_local_namespace
exec(codeString,
a_global_namespace)
exec(codeString,
a_global_namespace,
a_local_namespace)
exec()
2to3
2.
Python2 exec
Python3
exec()
Python2 exec
3.
() Python3
exec()
EXECFILE
execPython2 execfile
Pythonexec
execfilePython3 execfile
Python(
)
compile()Python
exec()
Notes Python2
Python3
execfile('a_filename') exec(compile(open('a_filename').read(),
'a_filename','exec'))
REPR()
Python2
(`x`) Python3
repr()
Notes
1.
Python2
`x`
Python3
repr(x)
`'PapayaWhip'+`2``
repr('PapayaWhip'
+repr(2))
x
repr()
2.
Python2
()2to3
repr()
TRY...EXCEPT
Python2 Python3
Notes
Python2
Python3
try:
try:
importmymodule
import
exceptImportError,e
mymodule
pass
except
ImportErroras
e:
pass
try:
try:
importmymodule
import
except(RuntimeError,
mymodule
ImportError),e
except
pass
(RuntimeError,
ImportError)
ase:
pass
try:
nochange
importmymodule
exceptImportError:
pass
try:
nochange
importmymodule
except:
1.
pass
Python2 Python3
as
2.
as
3.
Python2 Python3
4.
(fallback)
Python2 Python3
()
(
fallback)
KeyboardInterrupt(
CtrlC )
RAISE
Python3
Notes
Python2
raise
MyException
raise
MyException,
'error
message'
raise
MyException,
'error
message',
a_traceback
raise'error
message'
Python3
unchanged
raiseMyException('errormessage')
raiseMyException('error
message').with_traceback(a_traceback)
unsupported
1.
2.
Python2
Python3
3.
Python2
(stacktrace) Python3
4.
Python2
Python3
2to3
THROW
Python2 throw()
a_generator.throw()
Python3
Python3
Notes Python2
a_generator.throw(MyException) nochange
a_generator.throw(MyException, a_generator.throw(MyException('erro
'errormessage')
message'))
a_generator.throw('error
unsupported
message')
1.
Python2 Python3
2.
(errorstring)
3.
Python2 Python3
2to3
XRANGE()
Python2
range() range()
Python3 range()xrange()
Notes
Python2
Python3
xrange(10)
range(10)
a_list=range(10)
a_list=
list(range(10))
[iforiin
range(10)]
[iforiin
xrange(10)]
foriinrange(10):
nochange

1.
sum(range(10))
nochange
2to3 xrange()
range()
2.
Python2 range()2to3
2to3
list() range()
3.
xrange()
4.
for
5.
sum() 2to3
(view)
min()max()sum()list()tuple()
set()sorted()any()all()
RAW_INPUT()INPUT()
Python2
input() Python (
) raw_input()
Python
(wart)Python3 raw_input()
input() input()
Notes
Python2
raw_input()
Python3
input()
raw_input('prompt')
input('prompt')
input()
eval(input())
1.
raw_input() input()
2.
Python2 raw_input()
Python3
3.
Python
input() eval()
FUNC_*
Python2
Python3
Notes
Python2
Python3
a_function.func_name
a_function.__name__
a_function.func_doc
a_function.__doc__
a_function.func_defaults a_function.__defaults__
a_function.func_dict
a_function.__dict__
a_function.func_closure
a_function.__closure__
a_function.func_globals
a_function.__globals__
a_function.func_code
a_function.__code__
1.
__name__( func_name)
2.
__doc__( funcdoc)
(docstring)
3.
__defaults__( func_defaults)
4.
__dict__( func_dict)
5.
__closure__( func_closure) cell
(freevariable)
6.
__globals__( func_globals)
7.
__code__( func_code)
I/OXREADLINES()
Python2 xreadlines()
for
Python2
Python3 xreadlines()2to3
Notes
Python2
forlinein
a_file.xreadlines():
for
linein
a_file:
forlinein
a_file.xreadlines(5):
no
change
(broken)
1.
Python
3
xreadlines()2to3
Python3
for
2.
()
xreadlines()2to3 Python2 Python3
AttributeError:
'_io.TextIOWrapper'objecthasnoattribute
'xreadlines' xreadlines() readlines()
Python3 (readline() Python
3 Python2 xreadlines()
LAMBDA
Python2 lambda (anonymous
lambdafunction)
Python2
(unpack)(named
arguments) lambda (
) Python3 lambda
Python
(positionalindex)
Notes
Python2
Python3
lambda(x,):x+f(x)
lambdax1:
x1[0]+
f(x1[0])
lambdax_y:
x_y[0]+
f(x_y[1])
lambda(x,y):x+f(y)
lambda(x,(y,z)):x+
y+z
lambda
x_y_z:
x_y_z[0]+
x_y_z[1][0]
+
x_y_z[1][1]
lambdax,y,z:x+y+
z
unchanged
1.
lambda
Python3
x1[0] lambda x1 2to3
2.
(x,y) lambda
x_y x_y[0] x_y[1]
3.
2to3
lambda
Python3 Python2
4.
lambda
Python2
lambda lambda
Python
3
Python2 (class
object)(methodobject)im_self
im_func im_class
Python3
Notes
Python2
Python3
aClassInstance.aClassMethod.im_func aClassInstance.aClassMethod._
aClassInstance.aClassMethod.im_self aClassInstance.aClassMethod._
aClassInstance.aClassMethod.im_class aClassInstance.aClassMethod._
__NONZERO__
Python2
(booleancontext)
if
__nonzero__()
True False
Python3
__bool__()
Notes
Python2
Python3
classA:
classA:
def
def
__nonzero__(self):
__bool__(self):
pass
pass
classA:
nochange
def
__nonzero__(self,x,
y):
pass
Python3
1.
__bool__()__nonzero__()
__nonzero__()
2.
2to3
Python2 Python3 (octal)
Notes
SYS.MAXINT
Python2
x=0755
Python
3
x=
0o755
sys.maxint
Python3 sys.maxsize
Notes
Python2
Python3
fromsysimportmaxint fromsysimportmaxsize
a_function(sys.maxint) a_function(sys.maxsize)
1.
maxint maxsize
2.
sys.maxint sys.maxsize
CALLABLE()
Python2 callable()
(callable) Python3
__call__()
Notes
Python2
Python3
callable(anything)
hasattr(anything,
'__call__')
ZIP()
Python2 zip()
Python3 zip()
Notes
Python2
Python3
zip(a,b,c)
list(zip(a,
b,c))
d.join(zip(a,b,c))
nochange
list() zip()
1.
zip()list()
zip()
2.
join())zip()2to3
STANDARDERROR
Python2 StandardError StopIteration
GeneratorExitKeyboardInterruptSystemExit
Python3 StandardError
Exception
Notes
Python2
x=StandardError()
Python3
x=
Exception()
x=StandardError(a,b, x=
c)
Exception(a,
b,c)
TYPES
types
Python2
dict int Python3
Notes
Python2
types.UnicodeType
types.StringType
types.DictType
types.IntType
types.LongType
types.ListType
types.NoneType
types.BooleanType
types.BufferType
types.ClassType
types.ComplexType
types.EllipsisType
types.FloatType
types.ObjectType
types.NotImplementedType
types.SliceType
types.TupleType
types.TypeType
types.XRangeType
Python3
str
bytes
dict
int
int
list
type(None)
bool
memoryview
type
complex
type(Ellipsis)
float
object
type(NotImplemented)
slice
tuple
type
range
types.StringType bytes
str Python2
string( Unicode
)
(asequenceof
bytes)
ISINSTANCE()
isinstance()(class)
(type) Python2
(types) isinstance()
True Python3
Notes
Python2
Python3
isinstance(x,(int,
float,int))
isinstance(x,
(int,float))
BASESTRING
Python2 Unicode
Unicode
basestring str unicode
(superclass)
isinstance()
Unicode Unicode Python3
basestring
Notes
Python2
Python3
isinstance(x,
basestring)
isinstance(x,
str)
ITERTOOLS
Python2.3 itertoolszip()
map()filter()(variant)
Python3
itertools
(itertools
)
Notes
Python2
Python3
itertools.izip(a,b)
zip(a,b)
itertools.imap(a,b)
map(a,b)
itertools.ifilter(a,b)
filter(a,
b)
from
fromitertoolsimport
imap,izip,foo
itertools
import
foo
1.
zip() itertools.izip()
2.
map() itertools.imap()
3.
itertools.ifilter() filter()
4.
itertools Python3
2to3
SYS.EXC_TYPE, SYS.EXC_VALUE,
SYS.EXC_TRACEBACK
sys
sys.exc_typesys.exc_valuesys.exc_traceback(
Python1 ) Python1.5
sys.exc_info
Python3
sys.exc_info
Notes
Python2
Python3
sys.exc_type
sys.exc_value
sys.exc_traceback
sys.exc_info()[0]
sys.exc_info()[1]
sys.exc_info()[2]

Python2
Python3
Notes
Python
3
Python2
[iforiin1,2]
[ifor
iin
(1,
2)]
OS.GETCWDU()
Python2 os.getcwd()
(Unicode)
Python2.3
os.getcwdu()os.getcwdu()
UnicodePython3
(Unicode)os.getcwd()
Notes
(METACLASS)
Python2
Python3
os.getcwdu()
os.getcwd()
Python2 metaclass
(class
level)__metaclass__ Python3
__metaclass__
Notes
Python2
class
Python3
unchanged
C(metaclass=PapayaMeta):
1.
pass
classWhip:
class
__metaclass__=
Whip(metaclass=PapayaMeta):
PapayaMeta
pass
classC(Whipper,
classC(Whipper,Beater,
Beater):
metaclass=PapayaMeta):
__metaclass__=
pass
PapayaMeta
metaclass Python2
Python3
2.
__metaclass__ Python2
Python3
3.
2to3

(fixes)
Python
(haveavestedinterest)o
Python
2to3
SET()(LITERAL)()
Python2 (literalset)
set(a_sequence)Python3
(literalnotation)({})
{}
2to3 set()
2to3 fset_literal
Notes
Before
After
set([1,2,3])
set((1,2,3))
set([iforiin
a_sequence])
{1,2,3}
{1,2,3}
{iforiin
a_sequence}
BUFFER()()
C Python (buffer
interface) Python
() Python3 buffer()
memoryview()(
)
2to3 buffer()
2to3 fbuffer
Notes
Before
After
x=buffer(y)
x=
memoryview(y)
()
Python (indentingandoutdenting)
Python
Python
(
Python2 Python3 ) 2to3
2to3
2to3 fwscomma
Notes
Before
a,b
{a:b}
(COMMON IDIOMS)()
Python while1:
loop Python1(Python Python2.3
1 0
) Python
After
a,b
{a:
b}
2to3
2to3 fidioms
Notes
Before
After
while1:
whileTrue:
do_stuff()
do_stuff()
type(x)==T
type(x)isT
a_list=
isinstance(x,T)
isinstance(x,T)
a_list=
list(a_sequence)
sorted(a_sequence)
a_list.sort()
do_stuff(a_list)
do_stuff(a_list)
20019MarkPilgrim
:Python3
:
Myspecialtyisbeingrightwhenotherpeoplearewrong.
GeorgeBernardShaw
Python
__init__()
x=
MyClass()
repr(x)
Python
x.__init__()
x.__repr__()
1.
2.
3.
4.
5.
str(x)
x.__str__()
bytes(x)
x.__bytes__()
format(x,
x.__format__(format_spec)
format_spec)
__init__()
__new__()
__repr__()Python
print(x)__str__()
bytesPython3
format_spec
FormatSpecificationMiniLanguagePython
decimal.py__format__()
__iter__()
__next__()
iter(seq)
next(seq)
Python
seq.__iter__()
seq.__next__()
reversed(seq) seq.__reversed__()
__iter__()
2.
__next__()
__reversed__()
3.
1.
for
forxinseq:
print(x)
Python3seq.__iter__()
__next__()x__next__()
StopIterationfor
Python
x.my_property x.__getattribute__('my_property')
x.my_property x.__getattr__('my_property')
x.my_property x.__setattr__('my_property',
=value
value)
del
x.__delattr__('my_property')
x.my_property
dir(x)
x.__dir__()

1.
2.
3.
4.
5.
__getattribute__()
Python
__getattr__()Python
xcolor
x.colorx.__getattr__('color')
x.color
__setattr__()
__delattr__()
__getattr__()__getattribute__()
__dir__()dir(x)
__getattr()__color
dir(x)color__dir__()
color
__getattr__()__getattribute__()
classDynamo:
def__getattr__(self,key):
ifkey=='color':
return'PapayaWhip'
else:
raiseAttributeError
>>>dyn=Dynamo()
>>>dyn.color
'PapayaWhip'
>>>dyn.color='LemonChiffon'
>>>dyn.color
'LemonChiffon'
__getattr()__
'color'
2.
__getattr()__
AttributeError
NonePython
None
3.
dyncolor
__getattr__()
4.
dyn.colordyn.color
__getattr__()dyn.color
1.
__getattribute__()
classSuperDynamo:
def__getattribute__(self,key):
ifkey=='color':
return'PapayaWhip'
else:
raiseAttributeError
>>>dyn=SuperDynamo()
>>>dyn.color
'PapayaWhip'
>>>dyn.color='LemonChiffon'
>>>dyn.color
'PapayaWhip'
1.
2.
dyn.color__getattribute__()
dyn.colordyn.color,
__getattribute__()
__getattribute__()
__getattribute__()
__setattr__()
__getattribute__()Python
classRastan:
def__getattribute__(self,key):
raiseAttributeError
defswim(self):
pass
>>>hero=Rastan()
>>>hero.swim()
File"<stdin>",line3,in__getattribute__
AttributeError
AttributeError
__getattribute__()
2.
hero.swim()PythonRastanswim()
__getattribute__()
__getattribute__()
__getattribute__()AttributeError
1.
__call__()
Python
my_instance() my_instance.__call__()
zipfile
zipzip
decryptor
__init__()
map()
#excerptfromzipfile.py
class_ZipDecrypter:
.
.
.
def__init__(self,pwd):
self.key0=305419896
self.key1=591751049
self.key2=878082192
forpinpwd:
self._UpdateKeys(p)
def__call__(self,c):
assertisinstance(c,int)
k=self.key2|2
c=c^(((k*(k^1))>>8)&255)
self._UpdateKeys(c)
returnc
.
.
.
zd=_ZipDecrypter(pwd)
bytes=zef_file.read(12)
h=list(map(zd,bytes[0:12]))
1.
_ZipDecryptor
_UpdateKeys()
2.
__call__()
__call__()zip
3.
zd_ZipDecryptorpwd
__init__()
4.
zip12zd
__call__()12
12
Python
len(seq) seq.__len__()
xinseq seq.__contains__(x)
cgiFieldStorage
#Ascriptwhichrespondsto
http://example.com/search?q=cgi
importcgi
fs=cgi.FieldStorage()
if'q'infs:
do_search()
#Anexcerptfromcgi.pythatexplainshowthatworks
classFieldStorage:
.
.
.
def__contains__(self,key):
ifself.listisNone:
raiseTypeError('notindexable')
returnany(item.name==keyforitemin
self.list)
def__len__(self):
returnlen(self.keys())
1.
2.
3.
4.
5.
cgi.FieldStoragein
__contains__()
if'q'infsPythonfs
__contains__()cgi.py'q'
key__contains__()
FieldStorage
len(fs)FieldStorage__len__()
self.keys()self.listisNone
__len__
in len()
x[key]
x.__getitem__(key)
x[key]=value
x.__setitem__(key,value)
delx[key]
x.__delitem__(key)
Python
x[nonexistent_key] x.__missing__(nonexistent_key)
cgiFieldStorage
#Ascriptwhichrespondsto
http://example.com/search?q=cgi
importcgi
fs=cgi.FieldStorage()
if'q'infs:
do_search(fs['q'])
#Anexcerptfromcgi.pythatshowshowitworks
classFieldStorage:
.
.
.
def__getitem__(self,key):
ifself.listisNone:
raiseTypeError('notindexable')
found=[]
foriteminself.list:
ifitem.name==key:found.append(item)
ifnotfound:
raiseKeyError(key)
iflen(found)==1:
returnfound[0]
else:
returnfound
1.
fscgi.FieldStorage
fs['q']
fs['q']key'q'__getitem__()
2.
(self.list)
.name
Fraction
>>>fromfractionsimportFraction
>>>x=Fraction(1,3)
>>>x/3
Fraction(1,9)
&
x+y
xy
x*y
x/y
x//y
x%y
Python
x.__add__(y)
x.__sub__(y)
x.__mul__(y)
x.__truediv__(y)
x.__floordiv__(y)
x.__mod__(y)
divmod(x, x.__divmod__(y)
y)
x**y
x.__pow__(y)
x<<y
x.__lshift__(y)
x>>y
x.__rshift__(y)
and
xor
or
x&y
x.__and__(y)
x^y
x.__xor__(y)
x|y
x.__or__(y)
>>>fromfractionsimportFraction
>>>x=Fraction(1,3)
>>>1/x
Fraction(3,1)
x/3
x.__truediv__(3) Fraction__truediv__()
x/y
1.
2.
xy
yx
x/y
xy
y
x
x+y
xy
x*y
Python
y.__radd__(x)
y.__rsub__(x)
y.__rmul__(x)
&
and
xor
or
x/y
y.__rtruediv__(x)
x//y
y.__rfloordiv__(x)
x%y
y.__rmod__(x)
divmod(x, y.__rdivmod__(x)
y)
x**y
y.__rpow__(x)
x<<y
y.__rlshift__(x)
x>>y
y.__rrshift__(x)
x&y
y.__rand__(x)
x^y
y.__rxor__(x)
x|y
y.__ror__(x)
x/=3
x
+=
y
x
=y
x
*=
y
x
/=
y
x
//=
y
x
%=
y
x
**=
y
x
<<=
y
Python
x.__iadd__(y)
x.__isub__(y)
x.__imul__(y)
x.__itruediv__(y)
x.__ifloordiv__(y)
x.__imod__(y)
x.__ipow__(y)
x.__ilshift__(y)
and
xor
or
x
>>=
y
x
&=
y
x
^=
y
x
|=
y
x.__irshift__(y)
x.__iand__(y)
x.__ixor__(y)
x.__ior__(y)
Python
x/=yPython
x.__itruediv__(y)
NotImplemented
2.
x.__truediv__(y)
NotImplementedx
x=x/y
3.
y.__rtruediv__(x)
NotImplementedx
1.
__itruediv__()
Python
x
+x
abs(x)
~x
complex(x)
int(x)
float(x)
Python
x.__neg__()
x.__pos__()
x.__abs__()
x.__invert__()
x.__complex__()
x.__int__()
x.__float__()
PEP357
>=x
<=x
x0
round(x)
x.__round__()
round(x,n)
x.__round__(n)
math.ceil(x)
x.__ceil__()
math.floor(x) x.__floor__()
math.trunc(x) x.__trunc__()
a_list[x]
a_list[x.__index__()]
x==
y
x!=
y
x<
y
x<=
y
x>
y
x>=
y
if
x:
Python
x.__eq__(y)
x.__ne__(y)
x.__lt__(y)
x.__le__(y)
x.__gt__(y)
x.__ge__(y)
x.__bool__()
__lt__()
__gt__()Python
__lt__()Python
__lt__()
__eq()__x
<=yPython__lt__()
__eq()____le__()
PythonPython
picklingunpickling
pickling
picklepickle
pickling
pickling
copy.copy(x)
Python
x.__copy__()
copy.deepcopy(x)
x.__deepcopy__()
pickle.dump(x,
file)
x.__getstate__()
pickle.dump(x,
file)
x.__reduce__()
pickle.dump(x,
x.__reduce_ex__(protocol_version)
file,
protocol_version)
unpickling
unpickling
x=
x.__getnewargs__()
pickle.load(file)
x=
x.__setstate__()
pickle.load(file)
*Python
__getnewargs__()
__setstate__()
WITH
withwith
with
with
withfile
#excerptfromio.py:
def_checkClosed(self,msg=None):
Python
with x.__enter__()
x:
with x.__exit__()
x:
'''Internal:raiseanValueErroriffileisclosed
'''
ifself.closed:
raiseValueError('I/Ooperationonclosedfile.'
ifmsgisNoneelsemsg)
def__enter__(self):
'''Contextmanagementprotocol.Returnsself.'''
self._checkClosed()
returnself
'''Contextmanagementprotocol.Callsclose()'''
self.close()
__enter__()__exit__()
__enter__()
_checkClosed()
__enter__()selfwith
2.
3.
with
__exit__()self.close().
1.
__exit__()
with
__exit__()With
Python
x=MyClass() x.__new__()
delx
x.__del__()
x.__slots__()
hash(x)
x.__hash__()
x.color
type(x).__dict__['color'].__get__(x,
type(x))
x.color=
'PapayaWhip'
type(x).__dict__['color'].__set__(x,
'PapayaWhip')
your
class
delx.color
type(x).__dict__['color'].__del__(x)
isinstance(x, MyClass.__instancecheck__(x)
MyClass)
issubclass(C, MyClass.__subclasscheck__(C)
MyClass)
issubclass(C, MyABC.__subclasshook__(C)
MyABC)
Python__del__()
Python
Python
weakrefgc
zipfile
cgi
collections
math
pickle
copy
abc()
Python
PEP357:
PEP3119:
20019MarkPilgrim
:Python3
Goforthonyourpath,asitexistsonlythroughyourwalking.
St.AugustineofHippo(attributed)
ArielOrtiz
ArielOrtiz
PythonDavidMertz
Python
PythonAdamGomaa
Getters/Setters/FuxorsRyanTomayko
Pythonproperty()
HowTo:RaymondHettinger
Python:Python
DavidMertz
PythonMarkSummerfield
Python
&:
threading
multiprocessing
PythonJesseNoller
PythonGILDavidBeazley
PythonDavidMertzMichele
Simionato
PythonDavidMertz
MicheleSimionato
PythonDavidMertz
MicheleSimionato
DougHellmanPythonPython
PYTHON 3-
Python3
Python3
PythonPython3
Pythonpython3
Googlepython3
SourceForge:Python3
GitHub:python3(python3
)
BitBucket:python3(python3
)
20019MarkPilgrim

深入 Python 3 中文版

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

深入 Python 3 中文版

Uploaded by

Copyright:

Available Formats

:Python3

THE MINUS LEVEL

REMOVING ITEMS FROM A LIST: BONUS

U+4E2D UTF16 4E2D 2D

{1} format() suffix

STRING VS. BYTES

byte b'' bytes byte

Python3 .py UTF

search() None search()

Python pass JavaC

MORE HALTING, MORE FIRE

Pythonshell stdout stderr

foo lang lang en

XML lxml ElementTree

lxml ElementTree findall()

[<Element {http://www.w3.org/2005/Atom}author at eeba80>,

pickle dump() Python

Python1.x pickle (0)

Python HTTP httplib2

PythonHTTP ETag httplib2 .

httplib2 Http Http

httplib2 LastModifiedvalidator IfModified

request() response URL

(BOM) UTFn UTF8

EUCJP SHIFT_JIS SJISProber

__init__.py detect() chardet

NO MODULE NAMED CONSTANTS

NAME 'FILE' IS NOT DEFINED

CANT USE A STRING PATTERN ON A BYTES-LIKE

CAN'T CONVERT 'BYTES' OBJECT TO STR

UNSUPPORTED OPERAND TYPE(S) FOR +: 'INT'

ORD() EXPECTED STRING OF LENGTH 1, BUT INT

UNORDERABLE TYPES: INT() >= STR()

GLOBAL NAME 'REDUCE' IS NOT DEFINED

Python2 Unicode Python3

Python3 urllib2 urllib

__closure__( func_closure) cell

Python3 (readline() Python

Python2 Python3 (octal)

You might also like

init.py detect() chardet

closure( func_closure) cell