Working with Zip archives - zipfile

Source code: Lib/zipfile.py

The ZIP file format is a common archive and compression standard. 

Tools to create, read, write, append, and list a ZIP file. 

Any advanced use of this module will require an understanding of the format, as defined in PKZIP Application 

Note.

This module does not currently handle multi-disk ZIP files. 

It can handle ZIP files that use the ZIP64 extensions (that is ZIP files that are more than 4 GiB in size). It supports decryption of encrypted files in ZIP archives, but it currently cannot create an encrypted file. Decryption is extremely slow as it is implemented in native Python rather than C.

The module defines the following items:

exception zipfile.BadZipFile The error raised for bad ZIP files.

New in version 3.2.

exception zipfile.BadZipfile

Alias of BadZipFile, for compatibility with older Python versions.

Deprecated since version 3.2.

exception zipfile.LargeZipFile The error raised when a ZIP file would require ZIP64 functionality but that has not been enabled.

class zipfile.ZipFile

The class for reading and writing ZIP files. 

See section ZipFile Objects for constructor details.

class zipfile.Path

A pathlib-compatible wrapper for zip files. 

See section Path Objects for details.

New in version 3.8.

class zipfile.PyZipFile

Class for creating ZIP archives containing Python libraries.

class zipfile.ZipInfo(filename='NoName', date_time=(1980, 1, 1, 0, 0, 0))

Class used to represent information about a member of an archive. 

Instances of this class are returned by the getinfo() and infolist() methods of ZipFile objects. 

Most users of the zipfile module will not need to create these, but only use those created by this

module. filename should be the full name of the archive member, and date_time should be a tuple containing six fields which describe the time of the last modification to the file;

the fields are described in section ZipInfo Objects.

zipfile.is_zipfile(filename)

Returns True if filename is a valid ZIP file based on its magic number, otherwise returns

False. filename may be a file or file-like object too.

Changed in version 3.1: Support for file and file-like objects.

zipfile.ZIP_STORED

The numeric constant for an uncompressed archive member.

zipfile.ZIP_DEFLATED

The numeric constant for the usual ZIP compression method. This requires the zlib

module.

zipfile.ZIP_BZIP2

The numeric constant for the BZIP2 compression method. This requires the bz2 module.

New in version 3.3.

zipfile.ZIP_LZMA

The numeric constant for the LZMA compression method. This requires the lzma module.

New in version 3.3.

Note: The ZIP file format specification has included support for bzip2 compression

since 2001, and for LZMA compression since 2006. However, some tools (including

older Python releases) do not support these compression methods, and may either

refuse to process the ZIP file altogether, or fail to extract individual files.

See also:

PKZIP Application Note

Documentation on the ZIP file format by Phil Katz, the creator of the format and

algorithms used.

Info-ZIP Home Page

Information about the Info-ZIP project’s ZIP archive programs and development

libraries.

ZipFile Objects

class zipfile.ZipFile(file, mode='r', compression=ZIP_STORED, allowZip64=True,

compresslevel=None, *, strict_timestamps=True)

Open a ZIP file, where file can be a path to a file (a string), a file-like object or a path-like

object.

The mode parameter should be 

'r' to read an existing file, 

'w' to truncate and write a new file, 

'a' to append to an existing file,

'x' to exclusively create and write a new file. 

If mode is 'x' and file refers to an existing file, a FileExistsError will be raised.

 If mode is 'a' and file refers to an existing ZIP file, then additional files are added to it. 

If file does not refer to a ZIP file, then a new ZIP archive is appended to the file. This is

meant for adding a ZIP archive to another file (such as python.exe). 

If mode is 'a' and the file does not exist at all, it is created. 

If mode is 'r' or 'a', the file should be seekable.

compression is the ZIP compression method to use when writing the archive, and should

be ZIP_STORED, ZIP_DEFLATED, ZIP_BZIP2 or ZIP_LZMA; 

unrecognized values will cause NotImplementedError to be raised. 

If ZIP_DEFLATED, ZIP_BZIP2 or ZIP_LZMA is specified but the corresponding module (zlib, bz2 or lzma) is not available, RuntimeError is raised. 

The default is ZIP_STORED.

If allowZip64 is True (the default) zipfile will create ZIP files that use the ZIP64 extensions

when the zipfile is larger than 4 GiB. 

If it is false zipfile will raise an exception when the ZIP file would require ZIP64 extensions.

The compresslevel parameter controls the compression level to use when writing files to

the archive. When using ZIP_STORED or ZIP_LZMA it has no effect. When using

ZIP_DEFLATED integers 0 through 9 are accepted (see zlib for more information). 

When using ZIP_BZIP2 integers 1 through 9 are accepted (see bz2 for more information).

The strict_timestamps argument, when set to False, allows to zip files older than 1980-

01-01 at the cost of setting the timestamp to 1980-01-01. Similar behavior occurs with

files newer than 2107-12-31, the timestamp is also set to the limit.

If the file is created with mode 'w', 'x' or 'a' and then closed without adding any files

to the archive, the appropriate ZIP structures for an empty archive will be written to the

file.

ZipFile is also a context manager and therefore supports the with statement. In the

example, myzip is closed after the with statement’s suite is finished—even if an

exception occurs:

New in version 3.2: Added the ability to use ZipFile as a context manager.

Changed in version 3.3: Added support for bzip2 and lzma compression.

Changed in version 3.4: ZIP64 extensions are enabled by default.

Changed in version 3.5: Added support for writing to unseekable streams. Added support

for the 'x' mode.

Changed in version 3.6: Previously, a plain RuntimeError was raised for unrecognized

compression values.

with ZipFile('spam.zip', 'w') as myzip:

myzip.write('eggs.txt')

Changed in version 3.6.2: The file parameter accepts a path-like object.

Changed in version 3.7: Add the compresslevel parameter.

New in version 3.8: The strict_timestamps keyword-only argument

ZipFile.close()

Close the archive file. You must call close() before exiting your program or essential

records will not be written.

ZipFile.getinfo(name)

Return a ZipInfo object with information about the archive member name. Calling

getinfo() for a name not currently contained in the archive will raise a KeyError.

ZipFile.infolist()

Return a list containing a ZipInfo object for each member of the archive. The objects are

in the same order as their entries in the actual ZIP file on disk if an existing archive was

opened.

ZipFile.namelist()

Return a list of archive members by name.

ZipFile.open(name, mode='r', pwd=None, *, force_zip64=False)

Access a member of the archive as a binary file-like object. name can be either the name

of a file within the archive or a ZipInfo object. 

The mode parameter, if included, must be

'r' (the default) or 'w'. pwd is the password used to decrypt encrypted ZIP files.

open() is also a context manager and therefore supports the with statement:

With mode 'r' the file-like object (ZipExtFile) is read-only and provides the following

methods: read(), 

readline(),

 readlines(), 

seek(), 

tell(),

 __iter__(),

__next__(). 

These objects can operate independently of the ZipFile.

With mode='w', a writable file handle is returned, which supports the write() method.

While a writable file handle is open, attempting to read or write other files in the ZIP file

will raise a ValueError.

When writing a file, if the file size is not known in advance but may exceed 2 GiB, pass

force_zip64=True to ensure that the header format is capable of supporting large files. If

the file size is known in advance, construct a ZipInfo object with file_size set, and

use that as the name parameter.

Note: The open(), read() and extract() methods can take a filename or a ZipInfo object. 

You will appreciate this when trying to read a ZIP file that contains members with duplicate 

names with ZipFile('spam.zip') as myzip:

with myzip.open('eggs.txt') as myfile:

print(myfile.read())

Changed in version 3.6: Removed support of mode='U'. Use io.TextIOWrapper for

reading compressed text files in universal newlines mode.

Changed in version 3.6: open() can now be used to write files into the archive with the

mode='w' option.

Changed in version 3.6: Calling open() on a closed ZipFile will raise a ValueError.

Previously, a RuntimeError was raised.

ZipFile.extract(member, path=None, pwd=None)

Extract a member from the archive to the current working directory; member must be its

full name or a ZipInfo object. Its file information is extracted as accurately as possible.

path specifies a different directory to extract to. member can be a filename or a ZipInfo

object. pwd is the password used for encrypted files.

Returns the normalised path created (a directory or new file).

Note: If a member filename is an absolute path, a drive/UNC sharepoint and leading

(back)slashes will be stripped, e.g.: ///foo/bar becomes foo/bar on Unix, and

C:\foo\bar becomes foo\bar on Windows. And all ".." components in a member

filename will be removed, e.g.: ../../foo../../ba..r becomes foo../ba..r. On

Windows illegal characters (:, <, >, |, ", ?, and *) replaced by underscore (_).

Changed in version 3.6: Calling extract() on a closed ZipFile will raise a ValueError.

Previously, a RuntimeError was raised.

Changed in version 3.6.2: The path parameter accepts a path-like object.

ZipFile.extractall(path=None, members=None, pwd=None)

Extract all members from the archive to the current working directory. path specifies a

different directory to extract to. members is optional and must be a subset of the list

returned by namelist(). pwd is the password used for encrypted files.

Warning: Never extract archives from untrusted sources without prior inspection. It is

possible that files are created outside of path, e.g. members that have absolute

filenames starting with "/" or filenames with two dots "..". This module attempts to

prevent that. See extract() note.

Changed in version 3.6: Calling extractall() on a closed ZipFile will raise a

ValueError. Previously, a RuntimeError was raised.

Changed in version 3.6.2: The path parameter accepts a path-like object.

ZipFile.printdir()

Print a table of contents for the archive to sys.stdout.

ZipFile.setpassword(pwd)

Set pwd as default password to extract encrypted files.

ZipFile.read(name, pwd=None)

Return the bytes of the file name in the archive. name is the name of the file in the

archive, or a ZipInfo object. The archive must be open for read or append. pwd is the

password used for encrypted files and, if specified, it will override the default password

set with setpassword(). Calling read() on a ZipFile that uses a compression method

other than ZIP_STORED, ZIP_DEFLATED, ZIP_BZIP2 or ZIP_LZMA will raise a

NotImplementedError. An error will also be raised if the corresponding compression

module is not available.

Changed in version 3.6: Calling read() on a closed ZipFile will raise a ValueError.

Previously, a RuntimeError was raised.

ZipFile.testzip()

Read all the files in the archive and check their CRC’s and file headers. Return the name

of the first bad file, or else return None.

Changed in version 3.6: Calling testzip() on a closed ZipFile will raise a ValueError.

Previously, a RuntimeError was raised.

ZipFile.write(filename, arcname=None, compress_type=None, compresslevel=None)

Write the file named filename to the archive, giving it the archive name arcname (by

default, this will be the same as filename, but without a drive letter and with leading path

separators removed). 

If given, compress_type overrides the value given for the compression parameter to the constructor for the new entry. Similarly, compresslevel will override the constructor if given. The archive must be open with mode 'w', 'x' or 'a'.

Note: Archive names should be relative to the archive root, that is, they should not

start with a path separator.

Note: If arcname (or filename, if arcname is not given) contains a null byte, the name

of the file in the archive will be truncated at the null byte.

Changed in version 3.6: Calling write() on a ZipFile created with mode 'r' or a closed

ZipFile will raise a ValueError. Previously, a RuntimeError was raised.

ZipFile.writestr(zinfo_or_arcname, data, compress_type=None, compresslevel=None)

Write a file into the archive. The contents is data, which may be either a str or a bytes

instance; if it is a str, it is encoded as UTF-8 first. zinfo_or_arcname is either the file

name it will be given in the archive, or a ZipInfo instance. 

If it’s an instance, at least the filename, date, and time must be given. 

If it’s a name, the date and time is set to the current date and time. The archive must be opened with mode 'w', 'x' or 'a'.

If given, compress_type overrides the value given for the compression parameter to the

constructor for the new entry, or in the zinfo_or_arcname (if that is a ZipInfo instance).

Similarly, compresslevel will override the constructor if given.

Note: When passing a ZipInfo instance as the zinfo_or_arcname parameter, the

compression method used will be that specified in the compress_type member of the

given ZipInfo instance. By default, the ZipInfo constructor sets this member to

ZIP_STORED.

Changed in version 3.2: The compress_type argument.

Changed in version 3.6: Calling writestr() on a ZipFile created with mode 'r' or a

closed ZipFile will raise a ValueError. Previously, a RuntimeError was raised.

The following data attributes are also available:

ZipFile.filename

Name of the ZIP file.

ZipFile.debug

The level of debug output to use. This may be set from 0 (the default, no output) to 3 (the

most output). Debugging information is written to sys.stdout.

ZipFile.comment

The comment associated with the ZIP file as a bytes object. If assigning a comment to a

ZipFile instance created with mode 'w', 'x' or 'a', it should be no longer than 65535

bytes. Comments longer than this will be truncated.

Path Objects

class zipfile.Path(root, at='')

Construct a Path object from a root zipfile (which may be a ZipFile instance or file

suitable for passing to the ZipFile constructor).

at specifies the location of this Path within the zipfile, e.g. ‘dir/file.txt’, ‘dir/’, or ‘’. Defaults

to the empty string, indicating the root.

Path objects expose the following features of pathlib.Path objects:

Path objects are traversable using the / operator.

Path.name

The final path component.

Path.open(*, **)

Invoke ZipFile.open() on the current path. Accepts the same arguments as

ZipFile.open().

Caution: The signature on this function changes in an incompatible way in Python

3.9. For a future-compatible version, consider using the third-party zipp.Path package

(3.0 or later).

Path.iterdir()

Enumerate the children of the current directory.

Path.is_dir()

Return True if the current context references a directory.

Path.is_file()

Return True if the current context references a file.

Path.exists()

Return True if the current context references a file or directory in the zip file.

Path.read_text(*, **)

Read the current file as unicode text. Positional and keyword arguments are passed

through to io.TextIOWrapper (except buffer, which is implied by the context).

Path.read_bytes()

Read the current file as bytes.

PyZipFile Objects

The PyZipFile constructor takes the same parameters as the ZipFile constructor, and one

additional parameter, optimize.

class zipfile.PyZipFile(file, mode='r', compression=ZIP_STORED,

allowZip64=True, optimize=-1)

New in version 3.2: The optimize parameter.

Changed in version 3.4: ZIP64 extensions are enabled by default.

Instances have one method in addition to those of ZipFile objects:

writepy(pathname, basename='', filterfunc=None)

Search for files *.py and add the corresponding file to the archive.

If the optimize parameter to PyZipFile was not given or -1, the corresponding file is

a *.pyc file, compiling if necessary.

If the optimize parameter to PyZipFile was 0, 1 or 2, only files with that optimization

level (see compile()) are added to the archive, compiling if necessary.

If pathname is a file, the filename must end with .py, and just the (corresponding

*.pyc) file is added at the top level (no path information). If pathname is a file that

does not end with .py, a RuntimeError will be raised. If it is a directory, and the

directory is not a package directory, then all the files *.pyc are added at the top level.

If the directory is a package directory, then all *.pyc are added under the package

name as a file path, and if any subdirectories are package directories, all of these are

added recursively in sorted order.

basename is intended for internal use only.

filterfunc, if given, must be a function taking a single string argument. It will be passed

each path (including each individual full file path) before it is added to the archive. 

If filterfunc returns a false value, the path will not be added, and if it is a directory its

contents will be ignored. 

For example, if our test files are all either in test directories or start with the string test_, we can use a filterfunc to exclude them:

The writepy() method makes archives with file names like this:

New in version 3.4: The filterfunc parameter.

Changed in version 3.6.2: The pathname parameter accepts a path-like object.

Changed in version 3.7: Recursion sorts directory entries.

ZipInfo Objects

Instances of the ZipInfo class are returned by the getinfo() and infolist() methods of

ZipFile objects. Each object stores information about a single member of the ZIP archive.

There is one classmethod to make a ZipInfo instance for a filesystem file:

classmethod ZipInfo.from_file(filename, arcname=None, *,

strict_timestamps=True)

Construct a ZipInfo instance for a file on the filesystem, in preparation for adding it to a

zip file.

filename should be the path to a file or directory on the filesystem.

Lambdas

Small functions and the lambda expression

Little functions that act as predicates or that combine elements in some way.

If there’s a Python built-in or a module function that’s suitable, you don’t need to define a new function at all:

stripped_lines = [line.strip() for line in lines]

existing_files = filter(os.path.exists, file_list)

If the function you need doesn’t exist, you need to write it. One way to write small functions is to use the lambda expression. 

lambda takes a number of parameters and an expression combining these parameters, and creates an anonymous function that returns the value of the expression:

adder = lambda x, y: x+y

print_assign = lambda name, value: name + '=' + str(value)

An alternative is to just use the def statement and define a function in the usual way:

def adder(x, y):

    return x + y

def print_assign(name, value):

    return name + '=' + str(value)

Which alternative is preferable?

That’s a style question

One reason is that lambda is quite limited in the functions it can define. The result has to be computable as a single expression, which means you can’t have multiway if... elif... else comparisons or try... except statements. 

If you try to do too much in a lambda statement, you’ll end up with an overly complicated expression that’s hard to read. 

Quick, what’s the following code doing?

import functools

total = functools.reduce(lambda a, b: (0, a[1] + b[1]), items)[1]

You can figure it out, but it takes time to disentangle the expression to figure out what’s going on. 

Using a short nested def statements makes things a little bit better:

import functools

def combine(a, b):

    return 0, a[1] + b[1]

total = functools.reduce(combine, items)[1]

But it would be best of all if I had simply used a for loop:

total = 0

for a, b in items:

    total += b

Or the sum() built-in and a generator expression:

total = sum(b for a, b in items)

Many uses of functools.reduce() are clearer when written as for loops.

Fredrik Lundh once suggested the following set of rules for refactoring uses of lambda:

• Write a lambda function.

• Write a comment explaining what the heck that lambda does.

• Study the comment for a while, and think of a name that captures the essence of the comment.

• Convert the lambda to a def statement, using that name.

• Remove the comment.

How and What are Closures

#Closure 1
def outer_func():
    message = 'Hi' #Variable set in the outer func
    def inner_func():
        print(message) 
        # message variable is not in the  inner func, the func looks for a 
        #free variable in the next level up
    return inner_func()
# outfunc()
my_func= outer_func
# name the users function my_func to reference the outer func 
# of the closure 
#(making the closure available to a variety of functions)
my_func() 
# Adding the parenthesis converts the my_func function from a 
# variable holding 
# a function to a function that is calling the outer function 

Showing the initial nested function. The inner function doesn't have a local variable so it looks for a free variable on the next level up.

# Closure 2
# Following on from Closure 1 showing the closure being used for two variable 
#labelled functions
#
# Note the inner function is pulling the msg value from the outer argument, 
# the argument is taken from the parameter supplied by the external functions 
def outer_func(msg):
    def inner_func():
        print(msg)
    return inner_func()
hi_func = outer_func('Hi')
hello_func = outer_func('Hello')

Showing the outer function taking an argument. Again the inner function is looking to the outer function for an argument and retrieves the argument received by the outer function. This allows the closure to be reusable. Assigning custom function labels to the outer function allows for different parameters to be passed into the function. Giving different answers.

# Closure 3
# A lot more going on here to show the benefit of the closure
import logging
#This is to capture information of the function and arguments
logging.basicConfig(filename='test.log', level=logging.INFO)
def logger(func):
    # Here is the closure getting used to log rather than
    # be the composition function
	def log_func(*args):
		logging.info(f'Running {func.__name__} with arguments {args}')
		print(func(*args))
	return log_func
# Functional Parameters being actioned
# Functions to be monitored by logger
def add(x,y):
	return x+y
def mul(x,y):
	return x*y
# Preparing the parameter function for the logger
add_log = logger(add)
mul_log = logger(mul)
# Function calls with the parameters of the functions
add_log(3,3)
mul_log(4,5)
add_log(12,23)
mul_log(2,100)

The final Closure shows the closure being used for a purpose that is not the computation but for logging information about the computation. Extending the use of the Closure 2 to not only carry out multiple computation but give the ability to log what happened in each on a a log file (test.log)