Files and Directories

pathlib implements path operations using pathlib.PurePath and pathlib.Path objects. The os and os.path modules, on the other hand, provide functions that work at a low level with str and bytes, which is more in line with a procedural approach. We consider pathlib’s object-oriented style to be more readable and will therefore cover it in greater detail here.

See also

Reading and Writing Files

In Python, you can open and read a file using the pathlib.Path class and various built-in read operations:

pathlib.Path.read_text()

returns the decoded contents of the file pointed to by the pointer as a string.

pathlib.Path.write_text()

opens the specified file in text mode, writes to it and then closes the file. An existing file with the same name will be overwritten.

Note

You do not need to use a with statement, as the file is already opened using a context manager.

Tip

However, when opening, reading and writing a file, you should always specify the character encoding explicitly, as Python versions prior to 3.15 will otherwise select a platform-dependent default encoding. An incorrect encoding can, however, trigger an exception or result in garbled text:

 1>>> from pathlib import Path
 2>>> p = Path("docs", "save-data", "python.txt")
 3>>> p.write_text("🐍", encoding="utf-8")
 41
 5>>> p.read_text(encoding="cp1252")
 6Traceback (most recent call last):
 7  File "<python-input-3>", line 1, in <module>
 8    p.read_text(encoding="cp1252")
 9    ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
10  File "/Users/veit/Library/Application Support/uv/python/cpython-3.14.6-macos-aarch64-none/lib/python3.14/pathlib/__init__.py", line 788, in read_text
11    return f.read()
12           ~~~~~~^^
13  File "/Users/veit/Library/Application Support/uv/python/cpython-3.14.6-macos-aarch64-none/lib/python3.14/encodings/cp1252.py", line 23, in decode
14    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
15           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
16UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 2: character maps to <undefined>
Line 2:

The arguments passed to pathlib.Path are path segments, either as PosixPath or WindowsPath. In the previous example, you open a file that you assume is located at docs/save-data/python.txt, relative to where you are calling the function.

The following example specifies an absolute path – C:\Users\Veit\My Documents\python.txt:

2>>> p = Path("C:/", "Users", "Veit", "My Documents", "python.txt")

We can suppress this exception using the errors argument. The default value for errors is strict, which causes an exception to be raised.

Any faulty byte can be replaced with the corresponding Unicode replacement character (U+FFFD, �):

>>> p.read_text(encoding="cp1252", errors="replace")
'�'

Added in version 3.15: From Python 3.15 onwards, UTF-8 is used by default everywhere. Only once support for Python 3.14 and earlier versions is discontinued will specifying the encoding for UTF-8 files become optional.

Alternatively, errors can also be ignored:

>>> p.read_text(encoding="cp1252", errors="ignore")
'ðŸ'

Or you can replace the bytes with an escape character:

>>> p.read_text(encoding="cp1252", errors="backslashreplace")
'ðŸ\\x90\\x8d'

The pure numerical byte value is specified by the hexadecimal digits following \\x.

Note

The errors argument is only useful if decoding fails. Reading a file with the wrong encoding can raise a UnicodeDecodeError. However, it can also simply result in garbled text without any error message if decoding with the wrong encoding appears to succeed, meaning that seemingly valid characters are displayed instead of an exception being raised. The errors argument cannot fix garbled text.

If you prefer to use open() – whether with a context manager or otherwise – you can use the pathlib.Path.open() method of your pathlib.Path object instead:

1>>> with p.open(encoding="utf-8") as f:
2...     f.readline()
3...
4'🐍'
Line 1:

pathlib.Path.open() does not read anything from the file, but returns a file object that you can use to access the opened file. It keeps track of a file and how much of it has been read or written. All file operations in Python are carried out using file objects rather than file names.

Line 2:

The first call to readline() returns the first line of the file object, that is, everything up to and including the first line break, or the entire file if there is no line break in the file; the next call to readline() returns the second line, if it exists, and so on. When there is nothing left to read, readline() returns an empty string.

This behaviour of readline() makes it easy, for example, to determine the number of lines in a file:

>>> with p.open(encoding="utf-8") as f:
...     lc = 0
...     while f.readline() != "":
...         lc = lc + 1
...     print(lc)
...
1

A quicker way to count all the lines is to use the built-in readlines() method, which reads all the lines of a file and returns them as a list of strings, with one string per line:

>>> with p.open(encoding="utf-8") as f:
...     print(len(f.readlines()))
...
1

However, if you are counting all the lines in a large file, this method may cause the buffer to overflow, as the entire file is read in one go. It is also possible for the buffer to overflow when using readline() if you attempt to read a line from a large file that contains no line-break characters. To better handle such situations, both methods have an optional argument that controls the amount of data read at any one time. Another way to iterate through all the lines of a file is to treat the file object as an iterator within a for loop:

>>> with p.open(encoding="utf-8") as f:
...     lc = 0
...     for l in f:
...         lc = lc + 1
...     print(lc)
...
1

The advantage of this method is that the lines are read into memory as and when required, so there is no risk of running out of memory, even with large files. Another advantage of this method is that it is simpler and easier to read.

However, a potential problem with the read method can arise when working on Windows and macOS in text mode, if you use the open() command in text mode, meaning without appending a b. In text mode, macOS converts every \r to \n, whilst Windows converts \r\n pairs to \n. You can specify how line breaks are handled by using the newline parameter when opening the file and setting newline="\n", \r or \r\n, which ensures that only that character sequence is used as a line break:

>>> with p.open(encoding="utf-8", newline="\r\n") as f:
...     lc = 0
...

In this example, only \n is treated as a line break. However, if the file is opened in binary mode, the encoding and newline arguments are meaningless, as all bytes are returned exactly as they appear in the file:

>>> with p.open(mode="rb") as f:
...     print(len(f.readlines()))
...
1

Reading directories

pathlib.Path.iterdir()

If the path points to a directory, the path objects representing the directory’s contents are returned:

>>> p = Path("docs", "save-data")
>>> for child in p.iterdir():
...     child
...
PosixPath('docs/save-data/index.rst')
PosixPath('docs/save-data/minidom_example.py')
PosixPath('docs/save-data/pickle.rst')
PosixPath('docs/save-data/xml.rst')
PosixPath('docs/save-data/books.xml')
PosixPath('docs/save-data/files.rst')

The child objects are returned in any order, and the special entries . and .. are not included. If the path is not a directory or is otherwise inaccessible, an OSError is raised.

pathlib.Path.glob()

finds the specified relative pattern in the directory represented by this path and returns all matching files:

>>> sorted(p.glob("*.rst"))
[PosixPath('docs/save-data/files.rst'), PosixPath('docs/save-data/index.rst'), PosixPath('docs/save-data/pickle.rst'), PosixPath('docs/save-data/xml.rst')]

See also

Pattern language

pathlib.Path.rglob()

recursively matches the specified relative pattern. This is equivalent to calling the function with **/ before the pattern.

pathlib.Path.walk()

generates the filenames within a directory structure by traversing the structure either top-down or bottom-up. It returns a 3-tuple consisting of (dirpath, dirnames, filenames).

With the default setting for the optional argument top_down=True, the tuple for a directory is generated before the tuples for its subdirectories.

When follow_symlinks=True, symlinks are resolved and placed in dirnames and filenames according to their targets.

The following example displays the sizes of the files in a directory, ignoring __pycache__ directories:

>>> for root, dirs, files in p.walk():
...     print(
...         root,
...         "consumes",
...         sum((root / file).stat().st_size for file in files),
...         "bytes in",
...         len(files),
...         "non-directory files",
...     )
...     if "__pycache__" in dirs:
...         dirs.remove("__pycache__")
...
docs/save-data consumes 88417 bytes in 13 non-directory files
docs/save-data/sqlite consumes 35187 bytes in 19 non-directory files

The next example is a simple implementation of shutil.rmtree(), whereby the directory tree must be traversed from the bottom up, as pathlib.Path.rmdir() only allows a directory to be deleted once it is empty:

>>> for root, dirs, files in p.walk(top_down=False):
...     for name in files:
...         (root / name).unlink()
...     for name in dirs:
...         (root / name).rmdir()
...

Creating files and directories

pathlib.Path.touch()

creates a file at the specified path. The mode parameter can be used to specify the file mode and access flags. If the file already exists, the modification time is updated to the current time provided that exist_ok=True; otherwise, a FileExistsError is raised.

Note

pathlib.Path.open() or pathlib.Path.write_text() are also frequently used to create files.

pathlib.Path.mkdir()

creates a new directory at the specified path. The parameters mode and exist_ok function as described in pathlib.Path.touch().

If parents=True, any missing parent directories in the path are created as required with the default permissions. With the default setting of parents=False, however, a FileNotFoundError is raised.

Move, Copy and Delete

pathlib.Path.rename()

renames the file or directory to the specified destination and returns a new pathlib.Path instance that points to the destination. On Unix, provided the destination exists and is a file, it is simply overwritten; on Windows, FileExistsError is raised.

>>> myfile = Path("docs", "save-data", "myfile.txt")
>>> newfile = Path("docs", "newdir", "newfile.txt")
>>> myfile.rename(newfile)
PosixPath('docs/newdir/newfile.txt')

Added in version 3.14: Python 3.14 introduces the methods pathlib.Path.copy(), pathlib.Path.copy_into(), pathlib.Path.move() and pathlib.Path.move_into().

See also

Python 3.14 Changelog

Permissions and Ownership

pathlib.Path.owner()

returns the name of the person who owns the file. By default, symbolic links are followed; however, if you wish to determine who owns the symbolic link, add follow_symlinks=False. If the user ID (UID) of the file cannot be found, a KeyError is raised.

pathlib.Path.group()

returns the name of the group that owns the file. The behaviour with regard to symbolic links is the same as that of pathlib.Path.owner(). If the group ID (GID) of the file cannot be found, a KeyError is also raised.

pathlib.Path.chmod()

changes the file mode and permissions. By default, it follows symbolic links. To change the permissions of symbolic links, you can use follow_symlinks=False or pathlib.Path.lchmod().

Comparison with os and os.path

  • pathlib implements objects with pathlib.PurePath and pathlib.Path, while os and os.path work more procedurally with low-level str and bytes.

  • Many functions in os and os.path support paths relative to directory descriptors. These functions are not available in pathlib.

  • str and bytes, as well as parts of python3:os-os and os.path, are written in C and are very fast. pathlib, on the other hand, is written in Python and is often slower, but this does not always matter.

Despite the differences, many os functions can be translated into corresponding pathlib.Path or pathlib.PurePath functions:

os

pathlib

os.chmod()

pathlib.Path.chmod()

os.lstat()

pathlib.Path.lstat()

os.listdir()

pathlib.Path.iterdir()

os.getcwd()

pathlib.Path.cwd()

os.lchmod()

pathlib.Path.lchmod()

os.link()

pathlib.Path.hardlink_to()

os.mkdir(), os.makedirs()

pathlib.Path.mkdir()

os.path.abspath()

pathlib.Path.absolute()

os.path.basename()

pathlib.PurePath.name

os.path.dirname()

pathlib.PurePath.parent

os.path.exists()

pathlib.Path.exists()

os.path.expanduser()

pathlib.Path.expanduser()

os.path.isabs()

pathlib.PurePath.is_absolute()

os.path.isdir()

pathlib.Path.is_dir()

os.path.isfile()

pathlib.Path.is_file()

os.path.isjunction()

pathlib.Path.is_junction()

os.path.islink()

pathlib.Path.is_symlink()

os.path.ismount()

pathlib.Path.is_mount()

os.path.join()

pathlib.PurePath.joinpath()

os.path.realpath()

pathlib.Path.resolve()

os.path.relpath()

pathlib.PurePath.relative_to()

os.path.samefile()

pathlib.Path.samefile()

os.path.splitext()

pathlib.PurePath.stem, pathlib.PurePath.suffix

os.readlink()

pathlib.Path.readlink()

os.remove(), os.unlink()

pathlib.Path.unlink()

os.rename()

pathlib.Path.rename()

os.replace()

pathlib.Path.replace()

os.rmdir()

pathlib.Path.rmdir()

os.stat()

pathlib.Path.stat()

os.symlink()

pathlib.Path.symlink_to()

os.walk()

pathlib.Path.walk()

Checks

  • Uses the functions of the pathlib module to take a path to a file named example.log and create a new file path in the same directory for a file named example.log1.

  • Open a file my_file.txt and insert additional text at the end of the file. Which command would you use to open my_file.txt? Which command would you use to reopen the file and read it from the beginning?

  • If you look at the man page for the wc utility, you will see two command line options:

    -c

    counts the bytes in the file

    -m

    counts the characters, which in the case of some Unicode characters can be two or more bytes long

    Also, if a file is specified, our module should read from and process that file, but if no file is specified, it should read from and process stdin.

  • If a context manager is used in a script that reads and/or writes multiple files, which of the following approaches do you think would be best?

    1. Put the entire script in a block managed by a with statement.

    2. Use one with statement for all reads and another for all writes.

    3. Use a with statement every time you read or write a file, that is, for every line.

    4. Use a with statement for each file you read or write.

  • Archive *.txt files from the current directory in the archive directory as *.zip files with the current date as the file name.

    • Which modules do you need for this?

    • Write a possible solution.