
Input and Output
****************

There are several ways to present the output of a program; data can be
printed in a human-readable form, or written to a file for future use.
This chapter will discuss some of the possibilities.


Fancier Output Formatting
=========================

So far we've encountered two ways of writing values: *expression
statements* and the ``print()`` function.  (A third way is using the
``write()`` method of file objects; the standard output file can be
referenced as ``sys.stdout``. See the Library Reference for more
information on this.)

Often you'll want more control over the formatting of your output than
simply printing space-separated values.  There are two ways to format
your output; the first way is to do all the string handling yourself;
using string slicing and concatenation operations you can create any
layout you can imagine.  The standard module ``string`` contains some
useful operations for padding strings to a given column width; these
will be discussed shortly.  The second way is to use the
``str.format()`` method.

The ``string`` module contains a class Template which offers yet
another way to substitute values into strings.

One question remains, of course: how do you convert values to strings?
Luckily, Python has ways to convert any value to a string: pass it to
the ``repr()`` or ``str()`` functions.

The ``str()`` function is meant to return representations of values
which are fairly human-readable, while ``repr()`` is meant to generate
representations which can be read by the interpreter (or will force a
``SyntaxError`` if there is not equivalent syntax).  For objects which
don't have a particular representation for human consumption,
``str()`` will return the same value as ``repr()``.  Many values, such
as numbers or structures like lists and dictionaries, have the same
representation using either function.  Strings and floating point
numbers, in particular, have two distinct representations.

Some examples:

   >>> s = 'Hello, world.'
   >>> str(s)
   'Hello, world.'
   >>> repr(s)
   "'Hello, world.'"
   >>> str(0.1)
   '0.1'
   >>> repr(0.1)
   '0.10000000000000001'
   >>> x = 10 * 3.25
   >>> y = 200 * 200
   >>> s = 'The value of x is ' + repr(x) + ', and y is ' + repr(y) + '...'
   >>> print(s)
   The value of x is 32.5, and y is 40000...
   >>> # The repr() of a string adds string quotes and backslashes:
   ... hello = 'hello, world\n'
   >>> hellos = repr(hello)
   >>> print(hellos)
   'hello, world\n'
   >>> # The argument to repr() may be any Python object:
   ... repr((x, y, ('spam', 'eggs')))
   "(32.5, 40000, ('spam', 'eggs'))"

Here are two ways to write a table of squares and cubes:

   >>> for x in range(1, 11):
   ...     print(repr(x).rjust(2), repr(x*x).rjust(3), end=' ')
   ...     # Note use of 'end' on previous line
   ...     print(repr(x*x*x).rjust(4))
   ...
    1   1    1
    2   4    8
    3   9   27
    4  16   64
    5  25  125
    6  36  216
    7  49  343
    8  64  512
    9  81  729
   10 100 1000

   >>> for x in range(1, 11):
   ...     print('{0:2d} {1:3d} {2:4d}'.format(x, x*x, x*x*x))
   ...
    1   1    1
    2   4    8
    3   9   27
    4  16   64
    5  25  125
    6  36  216
    7  49  343
    8  64  512
    9  81  729
   10 100 1000

(Note that in the first example, one space between each column was
added by the way ``print()`` works: it always adds spaces between its
arguments.)

This example demonstrates the ``rjust()`` method of string objects,
which right-justifies a string in a field of a given width by padding
it with spaces on the left.  There are similar methods ``ljust()`` and
``center()``.  These methods do not write anything, they just return a
new string.  If the input string is too long, they don't truncate it,
but return it unchanged; this will mess up your column lay-out but
that's usually better than the alternative, which would be lying about
a value.  (If you really want truncation you can always add a slice
operation, as in ``x.ljust(n)[:n]``.)

There is another method, ``zfill()``, which pads a numeric string on
the left with zeros.  It understands about plus and minus signs:

   >>> '12'.zfill(5)
   '00012'
   >>> '-3.14'.zfill(7)
   '-003.14'
   >>> '3.14159265359'.zfill(5)
   '3.14159265359'

Basic usage of the ``str.format()`` method looks like this:

   >>> print('We are the {0} who say "{1}!"'.format('knights', 'Ni'))
   We are the knights who say "Ni!"

The brackets and characters within them (called format fields) are
replaced with the objects passed into the format method.  The number
in the brackets refers to the position of the object passed into the
format method.

   >>> print('{0} and {1}'.format('spam', 'eggs'))
   spam and eggs
   >>> print('{1} and {0}'.format('spam', 'eggs'))
   eggs and spam

If keyword arguments are used in the format method, their values are
referred to by using the name of the argument.

   >>> print('This {food} is {adjective}.'.format(
   ...       food='spam', adjective='absolutely horrible'))
   This spam is absolutely horrible.

Positional and keyword arguments can be arbitrarily combined:

   >>> print('The story of {0}, {1}, and {other}.'.format('Bill', 'Manfred',
                                                          other='Georg'))
   The story of Bill, Manfred, and Georg.

An optional ``':'`` and format specifier can follow the field name.
This also greater control over how the value is formatted.  The
following example truncates the Pi to three places after the decimal.

>>> import math
>>> print('The value of PI is approximately {0:.3f}.'.format(math.pi))
The value of PI is approximately 3.142.

Passing an integer after the ``':'`` will cause that field to be a
minimum number of characters wide.  This is useful for making tables
pretty.:

   >>> table = {'Sjoerd': 4127, 'Jack': 4098, 'Dcab': 7678}
   >>> for name, phone in table.items():
   ...     print('{0:10} ==> {1:10d}'.format(name, phone))
   ...
   Jack       ==>       4098
   Dcab       ==>       7678
   Sjoerd     ==>       4127

If you have a really long format string that you don't want to split
up, it would be nice if you could reference the variables to be
formatted by name instead of by position.  This can be done by simply
passing the dict and using square brackets ``'[]'`` to access the keys

   >>> table = {'Sjoerd': 4127, 'Jack': 4098, 'Dcab': 8637678}
   >>> print('Jack: {0[Jack]:d}; Sjoerd: {0[Sjoerd]:d}; '
             'Dcab: {0[Dcab]:d}'.format(table))
   Jack: 4098; Sjoerd: 4127; Dcab: 8637678

This could also be done by passing the table as keyword arguments with
the '**' notation.:

   >>> table = {'Sjoerd': 4127, 'Jack': 4098, 'Dcab': 8637678}
   >>> print('Jack: {Jack:d}; Sjoerd: {Sjoerd:d}; Dcab: {Dcab:d}'.format(**table))
   Jack: 4098; Sjoerd: 4127; Dcab: 8637678

This is particularly useful in combination with the new built-in
``vars()`` function, which returns a dictionary containing all local
variables.

For a complete overview of string formatting with ``str.format()``,
see *Format String Syntax*.


Old string formatting
---------------------

The ``%`` operator can also be used for string formatting. It
interprets the left argument much like a ``sprintf``-style format
string to be applied to the right argument, and returns the string
resulting from this formatting operation. For example:

   >>> import math
   >>> print('The value of PI is approximately %5.3f.' % math.pi)
   The value of PI is approximately 3.142.

Since ``str.format()`` is quite new, a lot of Python code still uses
the ``%`` operator. However, because this old style of formatting will
eventually removed from the language ``str.format()`` should generally
be used.

More information can be found in the *Old String Formatting
Operations* section.


Reading and Writing Files
=========================

``open()`` returns a file object, and is most commonly used with two
arguments: ``open(filename, mode)``.

   >>> f = open('/tmp/workfile', 'w')

The first argument is a string containing the filename.  The second
argument is another string containing a few characters describing the
way in which the file will be used.  *mode* can be ``'r'`` when the
file will only be read, ``'w'`` for only writing (an existing file
with the same name will be erased), and ``'a'`` opens the file for
appending; any data written to the file is automatically added to the
end.  ``'r+'`` opens the file for both reading and writing. The *mode*
argument is optional; ``'r'`` will be assumed if it's omitted.

Normally, files are opened in *text mode*, that means, you read and
write strings from and to the file, which are encoded in a specific
encoding (the default being UTF-8).  ``'b'`` appended to the mode
opens the file in *binary mode*: now the data is read and written in
the form of bytes objects.  This mode should be used for all files
that don't contain text.

In text mode, the default is to convert platform-specific line endings
(``\n`` on Unix, ``\r\n`` on Windows) to just ``\n`` on reading and
``\n`` back to platform-specific line endings on writing.  This
behind-the-scenes modification to file data is fine for text files,
but will corrupt binary data like that in ``JPEG`` or ``EXE`` files.
Be very careful to use binary mode when reading and writing such
files.


Methods of File Objects
-----------------------

The rest of the examples in this section will assume that a file
object called ``f`` has already been created.

To read a file's contents, call ``f.read(size)``, which reads some
quantity of data and returns it as a string or bytes object.  *size*
is an optional numeric argument.  When *size* is omitted or negative,
the entire contents of the file will be read and returned; it's your
problem if the file is twice as large as your machine's memory.
Otherwise, at most *size* bytes are read and returned. If the end of
the file has been reached, ``f.read()`` will return an empty string
(``''``).

   >>> f.read()
   'This is the entire file.\n'
   >>> f.read()
   ''

``f.readline()`` reads a single line from the file; a newline
character (``\n``) is left at the end of the string, and is only
omitted on the last line of the file if the file doesn't end in a
newline.  This makes the return value unambiguous; if ``f.readline()``
returns an empty string, the end of the file has been reached, while a
blank line is represented by ``'\n'``, a string containing only a
single newline.

   >>> f.readline()
   'This is the first line of the file.\n'
   >>> f.readline()
   'Second line of the file\n'
   >>> f.readline()
   ''

``f.readlines()`` returns a list containing all the lines of data in
the file. If given an optional parameter *sizehint*, it reads that
many bytes from the file and enough more to complete a line, and
returns the lines from that.  This is often used to allow efficient
reading of a large file by lines, but without having to load the
entire file in memory.  Only complete lines will be returned.

   >>> f.readlines()
   ['This is the first line of the file.\n', 'Second line of the file\n']

An alternative approach to reading lines is to loop over the file
object. This is memory efficient, fast, and leads to simpler code:

   >>> for line in f:
   ...     print(line, end='')
   ...
   This is the first line of the file.
   Second line of the file

The alternative approach is simpler but does not provide as fine-
grained control.  Since the two approaches manage line buffering
differently, they should not be mixed.

``f.write(string)`` writes the contents of *string* to the file,
returning the number of characters written.

   >>> f.write('This is a test\n')
   15

To write something other than a string, it needs to be converted to a
string first:

   >>> value = ('the answer', 42)
   >>> s = str(value)
   >>> f.write(s)
   18

``f.tell()`` returns an integer giving the file object's current
position in the file, measured in bytes from the beginning of the
file.  To change the file object's position, use ``f.seek(offset,
from_what)``.  The position is computed from adding *offset* to a
reference point; the reference point is selected by the *from_what*
argument.  A *from_what* value of 0 measures from the beginning of the
file, 1 uses the current file position, and 2 uses the end of the file
as the reference point.  *from_what* can be omitted and defaults to 0,
using the beginning of the file as the reference point.

   >>> f = open('/tmp/workfile', 'rb+')
   >>> f.write(b'0123456789abcdef')
   16
   >>> f.seek(5)     # Go to the 6th byte in the file
   5
   >>> f.read(1)
   b'5'
   >>> f.seek(-3, 2) # Go to the 3rd byte before the end
   13
   >>> f.read(1)
   b'd'

In text files (those opened without a ``b`` in the mode string), only
seeks relative to the beginning of the file are allowed (the exception
being seeking to the very file end with ``seek(0, 2)``).

When you're done with a file, call ``f.close()`` to close it and free
up any system resources taken up by the open file.  After calling
``f.close()``, attempts to use the file object will automatically
fail.

   >>> f.close()
   >>> f.read()
   Traceback (most recent call last):
     File "<stdin>", line 1, in ?
   ValueError: I/O operation on closed file

It is good practice to use the ``with`` keyword when dealing with file
objects.  This has the advantage that the file is properly closed
after its suite finishes, even if an exception is raised on the way.
It is also much shorter than writing equivalent ``try``-``finally``
blocks:

   >>> with open('/tmp/workfile', 'r') as f:
   ...     read_data = f.read()
   >>> f.closed
   True

File objects have some additional methods, such as ``isatty()`` and
``truncate()`` which are less frequently used; consult the Library
Reference for a complete guide to file objects.


The ``pickle`` Module
---------------------

Strings can easily be written to and read from a file. Numbers take a
bit more effort, since the ``read()`` method only returns strings,
which will have to be passed to a function like ``int()``, which takes
a string like ``'123'`` and returns its numeric value 123.  However,
when you want to save more complex data types like lists,
dictionaries, or class instances, things get a lot more complicated.

Rather than have users be constantly writing and debugging code to
save complicated data types, Python provides a standard module called
``pickle``. This is an amazing module that can take almost any Python
object (even some forms of Python code!), and convert it to a string
representation; this process is called *pickling*.  Reconstructing the
object from the string representation is called *unpickling*.  Between
pickling and unpickling, the string representing the object may have
been stored in a file or data, or sent over a network connection to
some distant machine.

If you have an object ``x``, and a file object ``f`` that's been
opened for writing, the simplest way to pickle the object takes only
one line of code:

   pickle.dump(x, f)

To unpickle the object again, if ``f`` is a file object which has been
opened for reading:

   x = pickle.load(f)

(There are other variants of this, used when pickling many objects or
when you don't want to write the pickled data to a file; consult the
complete documentation for ``pickle`` in the Python Library
Reference.)

``pickle`` is the standard way to make Python objects which can be
stored and reused by other programs or by a future invocation of the
same program; the technical term for this is a *persistent* object.
Because ``pickle`` is so widely used, many authors who write Python
extensions take care to ensure that new data types such as matrices
can be properly pickled and unpickled.
