Python 3 Standard Library by Example, The Developer's Library Series

Master the Powerful Python 3 Standard Library through Real Code Examples

?The genius of Doug?s approach is that with 15 minutes per week, any motivated programmer can learn the Python Standard Library. Doug?s guided tour will help you flip the switch to fully power-up Python?s batteries.?

?Raymond Hettinger, Distinguished Python Core Developer

The Python 3 Standard Library contains hundreds of modules for interacting with the operating system, interpreter, and Internet?all extensively tested and ready to jump-start application development. Now, Python expert Doug Hellmann introduces every major area of the Python 3.x library through concise source code and output examples. Hellmann?s examples fully demonstrate each feature and are designed for easy learning and reuse.

You?ll find practical code for working with text, data structures, algorithms, dates/times, math, the file system, persistence, data exchange, compression, archiving, crypto, processes/threads, networking, Internet capabilities, email, developer and language tools, the runtime, packages, and more. Each section fully covers one module, with links to additional resources, making this book an ideal tutorial and reference.

The Python 3 Standard Library by Example introduces Python 3.x?s new libraries, significant functionality changes, and new layout and naming conventions. Hellmann also provides expert porting guidance for moving code from 2.x Python standard library modules to their Python 3.x equivalents.

• Manipulate text with string, textwrap, re (regular expressions), and difflib
• Use data structures: enum, collections, array, heapq, queue, struct, copy, and more
• Implement algorithms elegantly and concisely with functools, itertools, and contextlib
• Handle dates/times and advanced mathematical tasks
• Archive and data compression
• Understand data exchange and persistence, including json, dbm, and sqlite
• Sign and verify messages cryptographically
• Manage concurrent operations with processes and threads
• Test, debug, compile, profile, language, import, and package tools
• Control interaction at runtime with interpreters or the environment

Introduction xxxi

Acknowledgments xxxiii

About the Author xxxv

Chapter 1: Text 1

1.1 string: Text Constants and Templates 1

1.2 textwrap: Formatting Text Paragraphs 7

1.3 re: Regular Expressions 13

1.4 difflib: Compare Sequences 58

Chapter 2: Data Structures 65

2.1 enum: Enumeration Type 66

2.2 collections: Container Data Types 75

2.3 array: Sequence of Fixed-Type Data 98

2.4 heapq: Heap Sort Algorithm 103

2.5 bisect: Maintain Lists in Sorted Order 109

2.6 queue: Thread-Safe FIFO Implementation 111

2.7 struct: Binary Data Structures 117

2.8 weakref: Impermanent References to Objects 121

2.9 copy: Duplicate Objects 130

2.10 pprint: Pretty-Print Data Structures 136

Chapter 3: Algorithms 143

3.1 functools: Tools for Manipulating Functions 143

3.2 itertools: Iterator Functions 163

3.3 operator: Functional Interface to Built-in Operators 183

3.4 contextlib: Context Manager Utilities 191

Chapter 4: Dates and Times 211

4.1 time: Clock Time 211

4.2 datetime: Date and Time Value Manipulation 221

4.3 calendar: Work with Dates 233

Chapter 5: Mathematics 239

5.1 decimal: Fixed- and Floating-Point Math 239

5.2 fractions: Rational Numbers 250

5.3 random: Pseudorandom Number Generators 254

5.4 math: Mathematical Functions 264

5.5 statistics: Statistical Calculations 290

Chapter 6: The File System 295

6.1 os.path: Platform-Independent Manipulation of Filenames 296

6.2 pathlib: File System Paths as Objects 305

6.3 glob: Filename Pattern Matching 319

6.4 fnmatch: Unix-Style Glob Pattern Matching 323

6.5 linecache: Read Text Files Efficiently 326

6.6 tempfile: Temporary File System Objects 330

6.7 shutil: High-Level File Operations 337

6.8 filecmp: Compare Files 351

6.9 mmap: Memory-Map Files 361

6.10 codecs: String Encoding and Decoding 365

6.11 io: Text, Binary, and Raw Stream I/O Tools 390

Chapter 7: Data Persistence and Exchange 395

7.1 pickle: Object Serialization 396

7.2 shelve: Persistent Storage of Objects 405

7.3 dbm: Unix Key–Value Databases 408

7.4 sqlite3: Embedded Relational Database 412

7.5 xml.etree.ElementTree: XML Manipulation API 445

7.6 csv: Comma-Separated Value Files 466

Chapter 8: Data Compression and Archiving 477

8.1 zlib: GNU zlib Compression 477

8.2 gzip: Read and Write GNU zip Files 486

8.3 bz2: bzip2 Compression 491

8.4 tarfile: Tar Archive Access 503

8.5 zipfile: ZIP Archive Access 511

Chapter 9: Cryptography 523

9.1 hashlib: Cryptographic Hashing 523

9.2 hmac: Cryptographic Message Signing and Verification 528

Chapter 10: Concurrency with Processes, Threads, and Coroutines 535

10.1 subprocess: Spawning Additional Processes 535

10.2 signal: Asynchronous System Events 553

10.3 threading: Manage Concurrent Operations Within a Process 560

10.4 multiprocessing: Manage Processes Like Threads 586

10.5 asyncio: Asynchronous I/O, Event Loop, and Concurrency Tools 617

10.6 concurrent.futures: Manage Pools of Concurrent Tasks 677

Chapter 11: Networking 687

11.1 ipaddress: Internet Addresses 687

11.2 socket: Network Communication 693

11.3 selectors: I/O Multiplexing Abstractions 724

11.4 select: Wait for I/O Efficiently 728

11.5 socketserver: Creating Network Servers 742

Chapter 12: The Internet 753

12.1 urllib.parse: Split URLs into Components 753

12.2 urllib.request: Network Resource Access 761

12.3 urllib.robotparser: Internet Spider Access Control 773

12.4 base64: Encode Binary Data with ASCII 776

12.5 http.server: Base Classes for Implementing Web Servers 781

12.6 http.cookies: HTTP Cookies 790

12.7 webbrowser: Displays Web Pages 796

12.8 uuid: Universally Unique Identifiers 797

12.9 json: JavaScript Object Notation 803

12.10 xmlrpc.client: Client Library for XML-RPC 816

12.11 xmlrpc.server: An XML-RPC Server 827

Chapter 13: Email 841

13.1 smtplib: Simple Mail Transfer Protocol Client 841

13.2 smtpd: Sample Mail Servers 847

13.3 mailbox: Manipulate Email Archives 852

13.4 imaplib: IMAP4 Client Library 864

Chapter 14: Application Building Blocks 887

14.1 argparse: Command-Line Option and Argument Parsing 888

14.2 getopt: Command-Line Option Parsing 916

14.3 readline: The GNU readline Library 922

14.4 getpass: Secure Password Prompt 935

14.5 cmd: Line-Oriented Command Processors 938

14.6 shlex: Parse Shell-Style Syntaxes 951

14.7 configparser: Work with Configuration Files 960

14.8 logging: Report Status, Error, and Informational Messages 980

14.9 fileinput: Command-Line Filter Framework 986

14.10 atexit: Program Shutdown Callbacks 993

14.11 sched: Timed Event Scheduler 998

Chapter 15: Internationalization and Localization 1003

15.1 gettext: Message Catalogs 1003

15.2 locale: Cultural Localization API 1012

Chapter 16: Developer Tools 1023

16.1 pydoc: Online Help for Modules 1024

16.2 doctest: Testing Through Documentation 1026

16.3 unittest: Automated Testing Framework 1051

16.4 trace: Follow Program Flow 1069

16.5 traceback: Exceptions and Stack Traces 1078

16.6 cgitb: Detailed Traceback Reports 1089

16.7 pdb: Interactive Debugger 1101

16.8 profile and pstats: Performance Analysis 1140

16.9 timeit: Time the Execution of Small Bits of Python Code 1148

16.10 tabnanny: Indentation Validator 1153

16.11 compileall: Byte-Compile Source Files 1155

16.12 pyclbr: Class Browser 1160

16.13 venv: Create Virtual Environments 1163

16.14 ensurepip: Install the Python Package Installer 1167

Chapter 17: Runtime Features 1169

17.1 site: Site-wide Configuration 1169

17.2 sys: System-Specific Configuration 1178

17.3 os: Portable Access to Operating System–Specific Features 1227

17.4 platform: System Version Information 1246

17.5 resource: System Resource Management 1251

17.6 gc: Garbage Collector 1254

17.7 sysconfig: Interpreter Compile-Time Configuration 1270

Chapter 18: Language Tools 1279

18.1 warnings: Non-fatal Alerts 1279

18.2 abc: Abstract Base Classes 1287

18.3 dis: Python Byte-Code Disassembler 1296

18.4 inspect: Inspect Live Objects 1311

Chapter 19: Modules and Packages 1329

19.1 importlib: Python’s Import Mechanism 1329

19.2 pkgutil: Package Utilities 1334

19.3 zipimport: Load Python Code from ZIP Archives 1344

Appendix A: Porting Notes 1351

A.1 References 1351

A.2 New Modules 1352

A.3 Renamed Modules 1352

A.4 Removed Modules 1354

A.5 Deprecated Modules 1355

A.6 Summary of Changes to Modules 1356

Appendix B: Outside of the Standard Library 1367

B.1 Text 1367

B.2 Algorithms 1367

B.3 Dates and Times 1368

B.4 Mathematics 1368

B.5 Data Persistence and Exchange 1368

B.6 Cryptography 1369

B.7 Concurrency with Processes, Threads, and Coroutines 1369

B.8 The Internet 1369

B.9 Email 1370

B.10 Application Building Blocks 1370

B.11 Developer Tools 1371

Index of Python Modules 1373

Index 1375

Doug Hellmann is currently employed by Red Hat to work on OpenStack. He is on the OpenStack Technical Committee and contributes to many aspects of the project. He has been programming in Python since version 1.4, and has worked on a variety of UNIX and non-UNIX platforms for projects in fields such as mapping, medical news publishing, banking, and data center automation. Doug is a Fellow of the Python Software Foundation and served as its Communications Director from 2010-2012. After a year as a regular columnist for Python Magazine, he served as Editor-in-Chief from 2008-2009. Between 2007 and 2011, Doug published the popular "Python Module of the Week" series on his blog, and an earlier version of this book (for Python 2), The Python Standard Library By Example (Addison-Wesley, 2011). He lives in Athens, Georgia.

• Teaches through concise, modular examples
• Covers text, data structures, algorithms, dates/times, math, files, data management, crypto, processes/threads, networking/Internet, email, developer tools, runtime, modules, packages, and more
• Fully reflects new Python 3 syntax, PSL’s new layout and naming conventions, and many new modules
• Includes porting notes summarizing Python 3.x PSL changes every developer needs to know about