„Batteries included“

In Python, a library can consist of several components, including built-in data types and constants that can be used without an import statement, such as Numbers and Lists, as well as some built-in Functions and Exceptions. The largest part of the library is an extensive collection of Modules. If you have Python installed, there are also several libraries available for you to use.

Managing data types

The standard library naturally contains support for the types built into Python. In addition, there are three categories in the standard library that deal with different data types: Modules for strings, datatypes and numbers.

String modules

:

Module

Description

string

compares with constants such as string.digits or string.whitespace

re

searches and replaces text with regular expressions

struct

interprets bytes as packed binary data

difflib

helps to calculate deltas, find differences between strings or sequences and create patches and diff files

textwrap

wraps and fills text, formats text with line breaks or spaces

See also

Checks

  • For example, can you add or multiply a string with an integer, a floating point number or a complex number?

  • How can you change a heading such as variables and expressions so that it contains hyphens instead of spaces and can therefore be better used as a file name?

  • Which of the following strings cannot be converted into numbers and why?

  • int("1e2")

  • int(1e+2)

  • int("1+2")

  • int("+2")

  • If you want to check whether a line begins with .. note::, which method would you use? Are there any other options?

  • Suppose you have a string with exclamation marks, quotation marks and line breaks. How can these be removed from the string?

  • How can you change all spaces and punctuation marks from a string to a hyphen (-)?

  • What use cases can you imagine in which the struct module would be useful for reading or writing binary data?

    • when reading and writing a binary file

    • when reading from an external interface, where the data should be stored exactly as it was transmitted

  • Which regular expression would you use to find strings that represent the numbers between -3 and +3?

  • Which regular expression would you use to find hexadecimal values?

Modules for data types

Module

Description

datetime, calendar

Time and calendar operations

collections

Container data types

enum

allows the creation of enumeration classes that bind symbolic names to constant values

array

Efficient arrays of numeric values

sched

Event scheduler

queue

Synchronised queue class

copy

Shallow and deep copy operations

pprint

prints Python data structures „pretty“.

typing

supports commenting code with hints about the types of objects, especially function parameters and return values

Modules for numbers

:

Module

Description

numbers

for numeric abstract base classes

math, cmath

for mathematical functions for real and complex numbers

decimal

for decimal fixed-point and floating-point arithmetic

statistics

for functions for calculating mathematical statistics

fractions

for rational numbers

random

for generating pseudo-random numbers and selections and for shuffling sequences

itertools

for functions that create iterators for efficient loops

functools

for higher-order functions and operations on callable objects

operator

for standard operators as functions

Checks

  • Create some number variables (integers, floating point numbers and complex numbers). Experiment a little with what happens when you perform operations with them, even across types.

  • Load the math module and try out some of the functions. Then load the cmath module and do the same.

  • How can you restore the functions of the math module?

  • Decide whether the following statements are true or false:

    • 1

    • 0

    • -1

    • [0]

    • 1 and 0

    • 1 > 0 or []

Changing files

:

Module

Description

os.path

performs common pathname manipulations

pathlib

manipulates pathnames

fileinput

iterates over multiple input files

filecmp

compares files and directories

tempfile

creates temporary files and directories

glob, fnmatch

use UNIX-like path and file name patterns

linecache

randomly accesses lines of text

shutil

performs higher level file operations

mimetypes

Assignment of file names to MIME types

pickle, shelve

enable Python object serialisation and persistence, see also The pickle module

csv

reads and writes CSV files

json

JSON encoder and decoder

sqlite3

provides a DB-API 2.0 interface for SQLite databases, see also The sqlite module

xml, xml.parsers.expat, xml.dom, xml.sax, xml.etree.ElementTree

reads and writes XML files, see also R:doc:../save-data/xml

html.parser, html.entities

Parsing HTML and XHTML

configparser

reads and writes Windows-like configuration files (.ini)

base64, binhex, binascii, quopri, uu

encodes/decodes files or streams

struct

reads and writes structured data to and from files

zlib, gzip, bz2, zipfile, tarfile

for working with archive files and compressions

See also

Checks

  • Uses the functions of the os module to take a path to a file named example.log and create a new file path in the same directory for a file named example.log1.

    >>> import os
>>> path = os.path.abspath("example.log")
>>> print(path)
/Users/veit/python-basics-tutorial-de/example.log
>>> new_path = f"{path}2"
>>> print(new_path)
/Users/veit/python-basics-tutorial-de/example.log2

  • What is the significance of adding b as a parameter to open()?

    This opens the file in binary mode, which means that bytes and not characters are read and written.

  • Open a file my_file.txt and insert additional text at the end of the file. Which command would you use to open my_file.txt? Which command would you use to reopen the file and read it from the beginning?

    >>> with open("my_file", "a") as f:
...     f.write("Hi, Pythinistas!\n")
...
17
>>> with open("my_file") as f:
...     print(f.readlines())
...
['Hi, Pythinistas!\n', 'Hi, Pythinistas!\n']

  • What use cases can you imagine in which the struct module would be useful for reading or writing binary data?

  • Why pickle may or may not be suitable for the following use cases:

    1. Saving some state variables from one run to the next

    2. Storing evaluation results

    3. Saving user names and passwords

    4. Saving a large dictionary with English terms

  • If you look at the man page for the wc utility, you will see two command line options:

    -c

    counts the bytes in the file

    -m

    counts the characters, which in the case of some Unicode characters can be two or more bytes long

    Also, if a file is specified, our module should read from and process that file, but if no file is specified, it should read from and process stdin.

  • If a context manager is used in a script that reads and/or writes multiple files, which of the following approaches do you think would be best?

    1. Put the entire script in a block managed by a with statement.

    2. Use one with statement for all reads and another for all writes.

    3. Use a with statement every time you read or write a file, that is, for every line.

    4. Use a with statement for each file you read or write.

  • Archive *.txt files from the current directory in the archive directory as *.zip files with the current date as the file name.

    • Which modules do you need for this?

    • Write a possible solution.

Interacting with the operating system

Module

Description

os

Various operating system interfaces

platform

Access to the identification data of the underlying platform

time

Time access and conversions

io

Tools for working with data streams

select

Waiting for I/O completion

optparse

Parser for command line options

curses

Terminal handling for character cell displays

getpass

Portable password entry

ctypes

provides C-compatible data types

threading

high-level threading interface

multiprocessing

Process-based threading interface

subprocess

Management of subprocesses

Use of Internet protocols

Module

description

socket, ssl

Low-level network interface and SSL wrapper for socket objects

email

Email and MIME processing package

mailbox

Manipulation of mailboxes in various formats

cgi, cgitb

Common Gateway Interface support

wsgiref

WSGI utilities and reference implementation

urllib.request, urllib.parse

Open and parse URLs

ftplib, poplib, imaplib, nntplib, smtplib, telnetlib

Clients for various Internet protocols

socketserver

Framework for network servers

http.server

HTTP server

xmlrpc.client, xmlrpc.server

XML-RPC client and server

Developing and debugging

Module

Description

pydoc

Documentation generator and online help system

doctest

Test examples from Python docstrings

unittest

Framework for unittests, see also Unittest

test.support

Utility functions for tests

trace

traces the execution of Python statements

pdb

Python debugger

logging

logging function for Python

timeit

measures the execution time of small code snippets

profile, cProfile

Python profiler

sys

System-specific parameters and functions

gc

Functions of the Python garbage collector

inspect

inspects objects live

atexit

exit handler

__future__

Future statement definitions

imp

allows access to the import internals

zipimport

imports modules from zip archives

modulefinder

finds modules used by a script