XXX Link into the standard docs

XXX ensure PSF license is incorporated in our book, much of the text of this chapter is the description of the builtin functions

Defining Functions and Using Built-Ins

Introduction

Functions are the fundamental unit of work in Python. In this chapter, we will start with the basics of functions. Then we look at using the builtin functions. These are the core functions that are always available, meaning they don’t require an explicit import into your namespace.

Next we will look at some alternative ways of defining functions, such as lambdas and classes. We will also look at more advanced types of functions, namely closures and generator functions.

As you will see, functions are very easy to define and use. Python encourages an incremental style of development that you can leverage when writing functions.

So how does this work out in practice? Often when writing a function it may make sense to start with a sequence of statements and just try it out in a console. Or maybe just write a short script in an editor. The idea is to just to prove a path and answer such questions as, “Does this API work in the way I expect?” Because top-level code in a console or script works just like it does in a function, it’s easy to later isolate this code in a function body and then package it as a function, maybe in a libary, or as a method as part of a class. The ease of doing this style of development is one aspect that makes Python such a joy to program in. And of course in the Jython implementation, it’s easy to do that within the context of any Java library.

Note

Perhaps the only tricky part is to keep the whitespace consistent as you change the identation level. The key then is to use a good editor that supports Python.

XX - JJ: This note sounds a bit negative on the whitespace that Python uses. In

my chapters I’ve been building the whitespace as a great feature of the language. As Mark Ramm had mentioned, our minds are wired just like Python is written…the whitespace is natural and allows for consistency and easy management.

And nearly everything else is in terms of functions, even what are typically declarations in other languages like Java. For example, a class definition or module import is just syntax around the underlying functions, which you can call yourself if you need to do so. (Incidentally, these functions are type and __import__ respectively, and you will be learning more about them later in the sections on builtins.)

XX - JJ: These topics are probably a bit advanced for a beginning programmer. I agree

that they are useful and should be included in the chapter…but perhaps not mentioned yet. An example right of the start would be great!

XXX Functions are first-class objects XXX incorporate

Function Syntax and Basics

Functions are usually defined by using the def keyword, the name of the function, its parameters (if any), and the body of code. We will start by looking at this example function:

def times2(n):
    return n * 2

XX - JJ: I like the times2 function as a starter…good choice here.

Normal usage can treat function definitions as being very simple. But there’s subtle power in every piece of the function definition, due to the fact that Python is a dynamic language. We look at these pieces from both a simple (the more typical case) and a more advanced perspective.

We will also look at some alternative ways of creating functions in a later section.

The def Keyword

Using def for define seems simple enough, and this keyword certainly can be used to declare a function just like you would in a static language. You should write most code that way in fact.

However, the more advanced usage is that a function definition can occur at any level in your code and be introduced at any time. Unlike the case in a language like C or Java, function definitions are not declarations. Instead they are executable statements. You can nest functions, and we’ll describe that more when we talk about nested scopes. And you can do things like conditionally define them.

This means it’s perfectly valid to write code like this:

if variant:
    def f():
        print "One way"
 else:
    def f():
        print "or another"

Please note, regardless of when and where the definition occurs, including its variants as above, the function definition will be compiled into a function object at the same time as the rest of the module or script that the function is defined in.

Naming the Function

We will describe this more in a later section, but the dir builtin function will tell us about the names defined in a given namespace, defaulting to the module, script, or console environment we are working in. With this new times2 function defined above, we now see the following (at least) in the console namespace:

>>> dir()
['__doc__', '__name__', 'times2']

We can also just look at what is bound to that name:

>>> times2
<function times2 at 0x1>

(This object is further introspectable. Try dir(times2) and go from there.)

We can also redefine a function at any time:

>>> def f(): print "Hello, world"
...
>>> def f(): print "Hi, world"
...
>>> f()
Hi, world

This is true not just of running it from the console, but any module or script. The original version of the function object will persist until it’s no longer referenced, at which point it will be ultimately be garbage collected. In this case, the only reference was the name f, so it became available for GC immediately upon rebind.

What’s important here is that we simply rebound the name. First it pointed to one function object, then another. We can see that in action by simply setting another name (equivalently, a variable) to times2:

>>> t2 = times2
>>> t2(5)
10

This makes passing a function as a parameter very easy, for a callback for example. But first, we need to look at function parameters in more detail.

Function Metaprogramming

A given name can only be associated with one function at a time, so can’t overload a function with multiple definitions. If you were to define two or more functions with the same name, the last one defined is used, as we saw.

However, it is possible to overload a function, or otherwise genericize it. You simply need to create a dispatcher function that then dispatches to your set of corresponding functions.

Function Parameters and Calling Functions

When defining a function, you specify the parameters it takes. Typically you will see something like the following. The syntax is familar:

XXX def f(a, b, c)

XX JJ: Maybe use the tip_calc function for explaining multiple parameters:

def tip_calc(amt, pct) or something similar? Although your example gets the point accross

Often defaults are specified:

XXXX def f(a, b=1, c=None)

With this being the general form of what it take:

XXX what's a clear way to describe this? probably from the python tutorial or ref
def f(param1[=default1], *args, **kwargs)

It is oftentimes nice to include default values when defining a function.  This

helps reduce the chance of errors when calling and using the function. For instance, if the tip_calc function were to be called without passing any arguments then no error would be raised if defaults were provided. However, if we were to try and do the same without any defaults thn an expected error would be raised. Let’s take a look at some examples to see how useful defaults can be.

First, let’s define the simple tip_calc function:

def tip_calc(amt, pct=.18):
    return amt * pct

Next, we’ll try to use the function in various ways. You will see that since we did not provide any default for the amt argument, we must specify a value for the it when calling the function.

# Call the function providing a value for both amt and pct arguments
>>> tip_calc(15.98, .18)
2.8764
# Call the function without providing a value for pct
>>> tip_calc(15.98)
2.8764
# Call the function without providing any values
>>> tip_calc()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: tip_calc() takes at least 1 argument (0 given)

What if a function is called specifying only one or two arguments when there are default values specified for each argument? How does the function know to which argument we wish to assign the passed values? Python allows the names of arguments to be specified when calling the function. Such a design helps one move past issues such as this.

def tip_calc(amt=1.00, pct=.18):
    return amt * .18

With this being the general form of what it take:

XXX what's a clear way to describe this? probably from the python tutorial or ref
def f(param1[=default1], *args, **kwargs)

Note

This is not exhaustive. You can also use tuple parameters, but in practice, they are not typically used, and were removed in Python 3. We recommend you don’t use them. For one thing, they cannot be properly introspected from Jython.

Calling a function is symmetric. You can call a function. The parentheses are mandatory.

Calling functions is also done by with a familiar syntax. For example, for the function x with parameters a,b,c that would be x(a,b,c). Unlike some other dynamic languages like Ruby and Perl, the use of parentheses is required syntax (due the function name being just like any other name).

Objects are strongly typed, as we have seen. But function parameters, like names in general in Python, are not typed. This means that any parameter can refer to any type of object.

We see this play out in the times2 function. The * operator not only means multiply for numbers, it also means repeat for sequences (like strings and lists). So you can use the times2 function as follows:

>>> times2(4)
8
>>> times2('abc')
'abcabc'
>>> times2([1,2,3])
[1, 2, 3, 1, 2, 3]

All parameters in Python are passed by reference. This is identical to how Java does it with object parameters. However, while Java does support passing unboxed primitive types by value, there are no such entities in Python. Everything is an object in Python.

Functions are objects too, and they can be passed as parameters:

XXX passing a function as a parameter - We can simply pass its name, then in the function using it

XX JJ: Try this example here:

# Define a function that takes two values and a mathematical function
>>> def perform_calc(value1, value2, func):
...     return func(value1, value2)
...
# Define a mathematical function to pass
>>> def mult_values(value1, value2):
...     return value1 * value2
...
>>> perform_calc(2, 4, mult_values)
8

# Define another mathematical function to pass
>>> def add_values(value1, value2):
...     return value1 + value2
...
>>> perform_calc(2, 4, add_values)
6
>>>

If you have more than two or so arguments, it often makes more sense to call a function by parameter, rather than by the defined order. This tends to create more robust code. So if you have a function draw_point(x,y), you might want to call it as draw_point(x=10,y=20).

Defaults further simplify calling a function. You use the form of param=default_value when defining the function. For instance, you might take our times2 function and generalize it:

def times_by(n, by=2):
    return n * by

This function is equivalent to times2 when called using that default value.

There’s one point to remember that oftens trips up developers. The default value is initialized exactly once, when the function is defined. That’s certainly fine for immutable values like numbers, strings, tuples, frozensets, and similar objects. But you need to ensure that if the default value is mutable, that it’s being used in this fashion correctly. So a dictionary for a shared cache makes sense. But this mechanism won’t work for but a list where we expect it is initialized to an empty list upon invocation. If you’re doing that, you need to write that explicitly in your code.

Lastly, a function can take an unspecified number of ordered arguments, through *args, and keyword args, through **kwargs. These parameter names (args and kwargs) are conventional, so you can use whatever name makes sense for your function. The markers * and ** are used to to determine that this functionality should be used.

XXX by factors

Calling Functions - Recursion

The code definition is separate from the name of the function. This distinction proves to be useful for decorators, as we will see later.

XXX Recursion. (I think it makes sense to not focus on recursion too much; it may be a fundamental aspect of computer science, but it’s also rarely necessary for most end-user software development. So let’s keep it in a sidebar.) Demo Fibonacci, since this requires no explanation, and it’s a non trivial use of recursion.

Note that Jython, like CPython, is ultimately stack based [at least until we have some tail call optimization support in JVM]. Recursion can be useful for expressing an algorithm compactly, but deeply recursive solutions on Jython can exhaust the JVM stack.

Memoization, as we will discuss with decorators, can make a recursive solution practical, however.

Function Body

Documenting Functions

First, you should specify a document string for the function. The docstring, if it exists, is a string that occurs as the first value of the function body:

def times2(n):
    """Given n, returns n * 2"""
    return n * 2

By convention, use triple-quoted strings, even if your docstring is not multiline. If it is multiline, this is how we recommend you format it:

def fact(n):
    """Returns the factorial of n

    Computes the factorial of n recursively. Does not check its
    arguments if nonnegative integer or if would stack
    overflow. Use with care!
    """

    if n in (0, 1):
        return 1
    else:
        return n * fact(n - 1)

Any such docstring, but with leading indendetation stripped, becomes the __doc__ attribute of that function object. Incidentally, docstrings are also used for modules and classes, and they work exactly the same way.

You can now use the help built-in function to get the docstring, or see them from various IDEs like PyDev for Eclipse and nbPython for NetBeans as part of the auto-complete:

XXX help(fact)

Returning Values

All functions return some value.

In

times2, we use the return statement to exit the function with that value.

Functions can easily return multiple values at once by returning a tuple or other structure:

XXX especially show the construct return x, y - this is an elegant way to do multiple values

XX JJ: A simple example returning two values

>>> def calc_tips(amount):
...     return (amount * .18), (amount * .20)
...
>>> calc_tips(25.25)
(4.545, 5.050000000000001)

A function can return at any time:

XXX

And it can also return any object as its value. So you can have a function that looks like this:

XXX think of an interesting, simple function that returns different values based on input

XX JJ: In this example, we rewrite the perform_calc function to accept only positive numbers

and return an error message if a negative number is passed into it.

   >>> def check_pos_perform_calc(num1, num2, func):
   ...     if num1 > 0 and num2 > 0:
   ...         return func(num1, num2)
   ...     else:
   ...         return 'Only positive numbers can be used with this function!'
   ...
   >>> def mult_values(value1, value2):
   ...     return value1 * value2
   ...
   >>> check_pos_perform_calc(3, 4, mult_values)
   12
   >>> check_pos_perform_calc(3, -44, mult_values)
   'Only positive numbers can be used with this function!'


If a return statement is not used, the value ``None`` is returned. There is no

equivalent to a void method in Java, because every function in Python returns a value. However, the Python console will not show the return value when it’s None, so you need to explicitly print it to see what is returned:

>>> do_nothing()
>>> print do_nothing()
None

A delighter in Python is the ease by which it enables returning multiple values:

XXX function - return a, b

XX JJ: Already covered this, no?

We can then readily unpack the return value.

Introducing Variables

XXX local variables - extend this with discussion XXX global variables

A function introduces a scope for new names, such as variables. Any names that are created in the function are only visible within that scope:

XXX scope

XX JJ: In the following example, the sq variable is defined within the

scope of the function definition itself. If we try to use it outside of the function then we’ll receive an error.

>>> def square_num(num):
...     sq = num * num
...     return sq
...
>>> square_num(35)
1225
>>> sq
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'sq' is not defined

(Example showing a syntax error…)

Global Variables

global keyword - [Useful for certain circumstances, certainly not core/essential, much like nonlocal in Py3K, so let’s not put too much focus on it.]

The global keyword is used to declare that a variable name is from the module scope (or script) containing this function. Using global is rarely necessary in practice, since it is not necessary if the name is called as a function or an attribute is accessed (through dotted notation).

This is a good example of where Python is providing a complex balancing between a complex idea - the lexical scoping of names, and the operations on them - and the fact that in practice it is doing the right thing.

XXX rewrite above, confusing

Functions create scopes for their variables. Assigning a variable, just like in a simple script, implicitly

XX JJ: I’d say that detail can be left out…the example we provided

above demonstrates function scope

Other Statements

Acceptable Statements

So what can go in a function body? Pretty much any statement, including material that we will cover later in this book. So you can define functions or classes or use even import, within the scope of that function.

In particular, performing a potentially expensive operation like import as last as possible, can reduce the startup time of your app. It’s even possible it will be never needed too.

There are a couple of exceptions to this rule. In both cases, these statements must go at the beginning of a module, similar to what we see in a static language like Java:

• Compiler directives. Python supports a limited set of compiler directives that have the provocative syntax of from __future__ import X; see PEP 236. These are features that will be eventually be made available, generally in the next minor revision (such as 2.5 to 2.6). In addition, it’s a popular place to put Easter eggs, such as from __future__ import braces. (Try it in the console, which also relaxes what it means to be performed at the beginning.)
• Source encoding declaration. Although technically not a statement – it’s in a specially parsed comment – this must go in the first or second line.

Empty Functions

An empty function still needs something in its body. You can use the pass statement:

def do_nothing():
    pass # here's how to specify an empty body of code

Or you can just have a docstring for the function body:

def empty_callback(*args, **kwargs):
    """Use this function where we need to supply a callback,
       but have nothing further to do.
    """

Why have a function that does nothing? As in math, it’s useful to have an operation that stands for doing nothing, like “add zero” or “multiply by one”. These identity functions eliminate special cases. Likewise, as see with empty_callback, we may need to specify a callback function when calling an API, but nothing actually needs to be done. By passing in an empty function – or having this be the default – we can simplify the API.

Miscellaneous

XXX various limits XXX currently limits of 64K java bytecode instructions when compiled. this will be relaxed in a future version

What do functions look like from Java?

They are instances of PyObject, supporting the __call__ method.

Additional introspection is available. If a function object is just a standard function written in Python, it will be of class PyFunction. A builtin function will be of class PyBuiltinFunction. But don’t assume that in your code, because many other objects support the function interface (__call__), and these potentially could be proxying, perhaps several layers deep, a given function. You can only assume it’s a PyObject.

XX JJ: This works in a sidebar, but is too much info for a beginner.

Builtin Functions

Builtin functions are those functions that are always in the Python namespace. In other words, these functions – and builtin exceptions, boolean values, and some other objects – are the only truly globally defined names. If you are familiar with Java, they are somewhat like the classes from java.lang.

Builtins are rarely sufficient, however; even a simple command line script generally needs to parse its arguments or read in from its standard input. So for this case you would need to import sys. And in the context of Jython, you will need to import the relevant Java classes you are using, perhaps with import java. But the builtin functions are really the core function that almost all Python code uses.

Constructor Functions

Constructor functions are used to create objects of a given type.

Note

In Python, the type is a constructor function; there’s no difference at all in Python. So you can use the type function, which we will discuss momentarily, to look up the type of an object, then make instances of that same type.

First we will look at the constructor functions, which are more typically used for conversion. This is because there is generally a convenient literal syntax available, or in the case of bool, there are only two such constants, True and False.

bool([x])

Convert a value to a Boolean, using the standard truth testing procedure. If x is false or omitted, this returns False; otherwise it returns True. bool is also a class, which is a subclass of int. Class bool cannot be subclassed further. Its only instances are False and True.

If no argument is given, this function returns False.

chr(i)

Return a string of one character whose ASCII code is the integer i. For example, chr(97) returns the string 'a'. This is the inverse of ord(). The argument must be in the range [0..255], inclusive; ValueError will be raised if i is outside that range. See also unichr().

complex([real[, imag]])

Create a complex number with the value real + imag*j or convert a string or number to a complex number. If the first parameter is a string, it will be interpreted as a complex number and the function must be called without a second parameter. The second parameter can never be a string. Each argument may be any numeric type (including complex). If imag is omitted, it defaults to zero and the function serves as a numeric conversion function like int(), long() and float(). If both arguments are omitted, returns 0j.

dict([arg])

Create a new data dictionary, optionally with items taken from arg. The dictionary type is described in typesmapping.

For other containers see the built in list, set, and tuple classes, and the collections module.

Although there is a convenient literal for creating dict objects:

a_dict = { 'alpha' : 1, 'beta' : 2, 'gamma' : 3 }

It can be more convenient to create them using the dict function:

a_dict = dict(alpha=1, beta=2, gamma=3)

Of course in this latter case, the keys of the entries being created must be valid Python keywords.

float([x])

Convert a string or a number to floating point. If the argument is a string, it must contain a possibly signed decimal or floating point number, possibly embedded in whitespace. The argument may also be [+|-]nan or [+|-]inf. Otherwise, the argument may be a plain or long integer or a floating point number, and a floating point number with the same value (within Python’s floating point precision) is returned. If no argument is given, returns 0.0.

frozenset([iterable])

Return a frozenset object, optionally with elements taken from iterable. The frozenset type is described in types-set.

For other containers see the built in dict, list, and tuple classes, and the collections module.

int([x[, radix]])

Convert a string or number to a plain integer. If the argument is a string, it must contain a possibly signed decimal number representable as a Python integer, possibly embedded in whitespace. The radix parameter gives the base for the conversion (which is 10 by default) and may be any integer in the range [2, 36], or zero. If radix is zero, the proper radix is determined based on the contents of string; the interpretation is the same as for integer literals. (See numbers.) If radix is specified and x is not a string, TypeError is raised. Otherwise, the argument may be a plain or long integer or a floating point number. Conversion of floating point numbers to integers truncates (towards zero). If the argument is outside the integer range a long object will be returned instead. If no arguments are given, returns 0.

The integer type is described in typesnumeric.

iter(o[, sentinel])

Return an iterator object. The first argument is interpreted very differently depending on the presence of the second argument. Without a second argument, o must be a collection object which supports the iteration protocol (the __iter__() method), or it must support the sequence protocol (the __getitem__() method with integer arguments starting at 0). If it does not support either of those protocols, TypeError is raised. If the second argument, sentinel, is given, then o must be a callable object. The iterator created in this case will call o with no arguments for each call to its next() method; if the value returned is equal to sentinel, StopIteration will be raised, otherwise the value will be returned.

New in version 2.2.

list([iterable])

Return a list whose items are the same and in the same order as iterable’s items. iterable may be either a sequence, a container that supports iteration, or an iterator object. If iterable is already a list, a copy is made and returned, similar to iterable[:]. For instance, list('abc') returns ['a', 'b', 'c'] and list( (1, 2, 3) ) returns [1, 2, 3]. If no argument is given, returns a new empty list, [].

list is a mutable sequence type, as documented in typesseq. For other containers see the built in dict, set, and tuple classes, and the collections module.

object()

Return a new featureless object. object is a base for all new style classes. It has the methods that are common to all instances of new style classes.

New in version 2.2.

Changed in version 2.3: This function does not accept any arguments. Formerly, it accepted arguments but ignored them.

open(filename[, mode[, bufsize]])

Open a file, returning an object of the file type described in section bltin-file-objects. If the file cannot be opened, IOError is raised. When opening a file, it’s preferable to use open() instead of invoking the file constructor directly.

The first two arguments are the same as for stdio’s :cfunc:`fopen`: filename is the file name to be opened, and mode is a string indicating how the file is to be opened.

The most commonly-used values of mode are 'r' for reading, 'w' for writing (truncating the file if it already exists), and 'a' for appending (which on some Unix systems means that all writes append to the end of the file regardless of the current seek position). If mode is omitted, it defaults to 'r'. The default is to use text mode, which may convert '\n' characters to a platform-specific representation on writing and back on reading. Thus, when opening a binary file, you should append 'b' to the mode value to open the file in binary mode, which will improve portability. (Appending 'b' is useful even on systems that don’t treat binary and text files differently, where it serves as documentation.) See below for more possible values of mode.

The optional bufsize argument specifies the file’s desired buffer size: 0 means unbuffered, 1 means line buffered, any other positive value means use a buffer of (approximately) that size. A negative bufsize means to use the system default, which is usually line buffered for tty devices and fully buffered for other files. If omitted, the system default is used. [#]_

Modes 'r+', 'w+' and 'a+' open the file for updating (note that 'w+' truncates the file). Append 'b' to the mode to open the file in binary mode, on systems that differentiate between binary and text files; on systems that don’t have this distinction, adding the 'b' has no effect.

In addition to the standard :cfunc:`fopen` values mode may be 'U' or 'rU'. Python is usually built with universal newline support; supplying 'U' opens the file as a text file, but lines may be terminated by any of the following: the Unix end-of-line convention '\n', the Macintosh convention '\r', or the Windows convention '\r\n'. All of these external representations are seen as '\n' by the Python program. If Python is built without universal newline support a mode with 'U' is the same as normal text mode. Note that file objects so opened also have an attribute called newlines which has a value of None (if no newlines have yet been seen), '\n', '\r', '\r\n', or a tuple containing all the newline types seen.

Python enforces that the mode, after stripping 'U', begins with 'r', 'w' or 'a'.

Python provides many file handling modules including fileinput, os, os.path, tempfile, and shutil.

ord(c)

Given a string of length one, return an integer representing the Unicode code point of the character when the argument is a unicode object, or the value of the byte when the argument is an 8-bit string. For example, ord('a') returns the integer 97, ord(u'\u2020') returns 8224. This is the inverse of chr() for 8-bit strings and of unichr() for unicode objects. If a unicode argument is given and Python was built with UCS2 Unicode, then the character’s code point must be in the range [0..65535] inclusive; otherwise the string length is two, and a TypeError will be raised.

range([start, ]stop[, step])

This is a versatile function to create lists containing arithmetic progressions. It is most often used in for loops.

However, we recommend the use of xrange instead.

The arguments must be plain integers. If the step argument is omitted, it defaults to 1. If the start argument is omitted, it defaults to 0. The full form returns a list of plain integers [start, start + step, start + 2 * step, ...]. If step is positive, the last element is the largest start + i * step less than stop; if step is negative, the last element is the smallest start + i * step greater than stop. step must not be zero (or else ValueError is raised). Example:

>>> range(10)
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> range(1, 11)
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
>>> range(0, 30, 5)
[0, 5, 10, 15, 20, 25]
>>> range(0, 10, 3)
[0, 3, 6, 9]
>>> range(0, -10, -1)
[0, -1, -2, -3, -4, -5, -6, -7, -8, -9]
>>> range(0)
[]
>>> range(1, 0)
[]

set([iterable])

Return a new set, optionally with elements are taken from iterable. The set type is described in types-set.

For other containers see the built in dict, list, and tuple classes, and the collections module.

New in version 2.4.

str([object])

Return a string containing a nicely printable representation of an object. For strings, this returns the string itself. The difference with repr(object) is that str(object) does not always attempt to return a string that is acceptable to eval(); its goal is to return a printable string. If no argument is given, returns the empty string, ''.

For more information on strings see typesseq which describes sequence functionality (strings are sequences), and also the string-specific methods described in the string-methods section. To output formatted strings use template strings or the % operator described in the string-formatting section. In addition see the stringservices section. See also unicode().

tuple([iterable])

Return a tuple whose items are the same and in the same order as iterable’s items. iterable may be a sequence, a container that supports iteration, or an iterator object. If iterable is already a tuple, it is returned unchanged. For instance, tuple('abc') returns ('a', 'b', 'c') and tuple([1, 2, 3]) returns (1, 2, 3). If no argument is given, returns a new empty tuple, ().

tuple is an immutable sequence type, as documented in typesseq. For other containers see the built in dict, list, and set classes, and the collections module.

type(name, bases, dict)

Return a new type object. This is essentially a dynamic form of the class statement. The name string is the class name and becomes the __name__ attribute; the bases tuple itemizes the base classes and becomes the __bases__ attribute; and the dict dictionary is the namespace containing definitions for class body and becomes the __dict__ attribute. For example, the following two statements create identical type objects:

>>> class X(object):
...     a = 1
...
>>> X = type('X', (object,), dict(a=1))

New in version 2.2.

unichr(i)

Return the Unicode string of one character whose Unicode code is the integer i. For example, unichr(97) returns the string u'a'. This is the inverse of ord() for Unicode strings. The valid range for the argument depends how Python was configured – it may be either UCS2 [0..0xFFFF] or UCS4 [0..0x10FFFF]. ValueError is raised otherwise. For ASCII and 8-bit strings see chr().

New in version 2.0.

unicode([object[, encoding[, errors]]])

Return the Unicode string version of object using one of the following modes:

If encoding and/or errors are given, unicode() will decode the object which can either be an 8-bit string or a character buffer using the codec for encoding. The encoding parameter is a string giving the name of an encoding; if the encoding is not known, LookupError is raised. Error handling is done according to errors; this specifies the treatment of characters which are invalid in the input encoding. If errors is 'strict' (the default), a ValueError is raised on errors, while a value of 'ignore' causes errors to be silently ignored, and a value of 'replace' causes the official Unicode replacement character, U+FFFD, to be used to replace input characters which cannot be decoded. See also the codecs module.

If no optional parameters are given, unicode() will mimic the behaviour of str() except that it returns Unicode strings instead of 8-bit strings. More precisely, if object is a Unicode string or subclass it will return that Unicode string without any additional decoding applied.

For objects which provide a __unicode__() method, it will call this method without arguments to create a Unicode string. For all other objects, the 8-bit string version or representation is requested and then converted to a Unicode string using the codec for the default encoding in 'strict' mode.

For more information on Unicode strings see typesseq which describes sequence functionality (Unicode strings are sequences), and also the string-specific methods described in the string-methods section. To output formatted strings use template strings or the % operator described in the string-formatting section. In addition see the stringservices section. See also str().

Use as decorators: classmethod, staticmethod, property

slice is rarely used directly.

super type - 3 arg form compile

Math Builtin Functions

Most math functions are defined in math (or cmath for complex math). These are functions that are builtin:

abs, cmp, divmod, pow, round

You may need to use named functions

Functions on Iterables

The next group of builtin functions operate on iterables, which in Jython also includes all Java objects that implement the java.util.Iterator interface.

In particular,

enumerate(iterable)

zip([iterable, ...])

The zip function creates a list of tuples by stepping through each iterable. One very common idiom is to use zip to create a dict where one iterable has the keys, and the other the values. This is often seen in working with CSV files (from a header row) or database cursors (from the description attribute). However, you might want to consider using collections.namedtuple instead:

XXX example code - read from CSV, zip together

sorted(iterable[, cmp[, key[, reverse]]])

The sorted function returns a sorted list. Use the optional key argument to specify a key function to control how it’s sorted. So for example, this will sort the list by the length of the elements in it:

>>> sorted(['Massachusetts', 'Colorado', 'New York', 'California', 'Utah'], key=len)
['Utah', 'Colorado', 'New York', 'California', 'Massachusetts']

And this one will sort a list of Unicode strings without regard to it whether the characters are upper or lowercase:

>>> sorted(['apple', 'Cherry', 'banana'])
['Cherry', 'apple', 'banana']

>>> sorted(['apple', 'Cherry', 'banana'], key=str.upper)
['apple', 'banana', 'Cherry']

Although using a key function requires building a decorated version of the list to be sorted, in practice this uses substantially less overhead than calling a cmp function on every comparison. We recommend you take advantage of a keyed sort.

all(iterable), any(iterable)

all and any will also short cut, if possible.

and sum(iterable[, start=0]) are functions that you will find frequent use for.

max(iterable[, key]) or max([, arg, ...][, key]); min(iterable[, key]) or min([, arg, ...][, key])

The max and min functions take a key function as an optional argument.

Although filter, map, and reduce are still useful, their use is largely superseded by using other functions, in conjunction with generator expressions. The range function is still useful for creating a list of a given sequence, but for portability eventualy to Python 3.x, using list(xrange()) instead is better.

Some advice:

• Generator expressions (or list comprehensions) are easier to use than filter.
• Most interesting but simple uses of reduce can be implemented through sum. And anything more complex should likely be written as a generator.

XXX some extra stuff here:

all(iterable)

Returns True if all of the elements in the iterable are true, otherwise False and stop the iteration. (If the iterable is empty, this function returns True).

any(iterable)

Returns True if any of the elements in the iterable are true, stopping the iteration. Otherwise returns False and stop the iteration. (If the iterable is empty, this function returns True).

Returns True if any of the

enumerate(iterable)

filter(function, iterable)

sum(iterable[, start=0])

XXX maybe show how to construct a count using bool

Namespace Functions

namespace - __import__, delattr, dir, getattr, locals, globals, hasattr, reload, setattr, vars

getattr

compile, eval, exec Creating code objects.

evaluation - eval, execfile, predicates - callable, isinstance, issubclass hex, oct, id, hash, ord, repr len input, rawinput

Just refer to the documentation on these: deprecated functions - apply, buffer, coerce, intern …

Operators

abs(x)

Return the absolute value of a number. The argument may be a plain or long integer or a floating point number. If the argument is a complex number, its magnitude is returned.

all(iterable)

Return True if all elements of the iterable are true. Equivalent to:

def all(iterable):
    for element in iterable:
        if not element:
            return False
    return True

New in version 2.5.

any(iterable)

Return True if any element of the iterable is true. Equivalent to:

def any(iterable):
    for element in iterable:
        if element:
            return True
    return False

New in version 2.5.

basestring()

This abstract type is the superclass for str and unicode. It cannot be called or instantiated, but it can be used to test whether an object is an instance of str or unicode. isinstance(obj, basestring) is equivalent to isinstance(obj, (str, unicode)).

New in version 2.3.

bin(x)

Convert an integer number to a binary string. The result is a valid Python expression. If x is not a Python int object, it has to define an __index__() method that returns an integer.

New in version 2.6.

callable(object)

Return True if the object argument appears callable, False if not. If this returns true, it is still possible that a call fails, but if it is false, calling object will never succeed. Note that classes are callable (calling a class returns a new instance); class instances are callable if they have a __call__() method.

classmethod(function)

Return a class method for function.

A class method receives the class as implicit first argument, just like an instance method receives the instance. To declare a class method, use this idiom:

class C:
    @classmethod
    def f(cls, arg1, arg2, ...): ...

The @classmethod form is a function decorator – see the description of function definitions in function for details.

It can be called either on the class (such as C.f()) or on an instance (such as C().f()). The instance is ignored except for its class. If a class method is called for a derived class, the derived class object is passed as the implied first argument.

Class methods are different than C++ or Java static methods. If you want those, see staticmethod() in this section.

For more information on class methods, consult the documentation on the standard type hierarchy in types.

New in version 2.2.

Changed in version 2.4: Function decorator syntax added.

cmp(x, y)

Compare the two objects x and y and return an integer according to the outcome. The return value is negative if x < y, zero if x == y and strictly positive if x > y.

compile(source, filename, mode[, flags[, dont_inherit]])

Compile the source into a code or AST object. Code objects can be executed by an exec statement or evaluated by a call to eval(). source can either be a string or an AST object. Refer to the ast module documentation for information on how to work with AST objects.

The filename argument should give the file from which the code was read; pass some recognizable value if it wasn’t read from a file ('<string>' is commonly used).

The mode argument specifies what kind of code must be compiled; it can be 'exec' if source consists of a sequence of statements, 'eval' if it consists of a single expression, or 'single' if it consists of a single interactive statement (in the latter case, expression statements that evaluate to something else than None will be printed).

The optional arguments flags and dont_inherit control which future statements (see PEP 236) affect the compilation of source. If neither is present (or both are zero) the code is compiled with those future statements that are in effect in the code that is calling compile. If the flags argument is given and dont_inherit is not (or is zero) then the future statements specified by the flags argument are used in addition to those that would be used anyway. If dont_inherit is a non-zero integer then the flags argument is it – the future statements in effect around the call to compile are ignored.

Future statements are specified by bits which can be bitwise ORed together to specify multiple statements. The bitfield required to specify a given feature can be found as the compiler_flag attribute on the _Feature instance in the __future__ module.

This function raises SyntaxError if the compiled source is invalid, and TypeError if the source contains null bytes.

Note

When compiling a string with multi-line statements, line endings must be represented by a single newline character ('\n'), and the input must be terminated by at least one newline character. If line endings are represented by '\r\n', use str.replace() to change them into '\n'.

Changed in version 2.3: The flags and dont_inherit arguments were added.

Changed in version 2.6: Support for compiling AST objects.

delattr(object, name)

This is a relative of setattr(). The arguments are an object and a string. The string must be the name of one of the object’s attributes. The function deletes the named attribute, provided the object allows it. For example, delattr(x, 'foobar') is equivalent to del x.foobar.

dir([object])

Without arguments, return the list of names in the current local scope. With an argument, attempt to return a list of valid attributes for that object.

If the object has a method named __dir__(), this method will be called and must return the list of attributes. This allows objects that implement a custom __getattr__() or __getattribute__() function to customize the way dir() reports their attributes.

If the object does not provide __dir__(), the function tries its best to gather information from the object’s __dict__ attribute, if defined, and from its type object. The resulting list is not necessarily complete, and may be inaccurate when the object has a custom __getattr__().

The default dir() mechanism behaves differently with different types of objects, as it attempts to produce the most relevant, rather than complete, information:

• If the object is a module object, the list contains the names of the module’s attributes.
• If the object is a type or class object, the list contains the names of its attributes, and recursively of the attributes of its bases.
• Otherwise, the list contains the object’s attributes’ names, the names of its class’s attributes, and recursively of the attributes of its class’s base classes.

The resulting list is sorted alphabetically. For example:

>>> import struct
>>> dir()   # doctest: +SKIP
['__builtins__', '__doc__', '__name__', 'struct']
>>> dir(struct)   # doctest: +NORMALIZE_WHITESPACE
['Struct', '__builtins__', '__doc__', '__file__', '__name__',
 '__package__', '_clearcache', 'calcsize', 'error', 'pack', 'pack_into',
 'unpack', 'unpack_from']
>>> class Foo(object):
...     def __dir__(self):
...         return ["kan", "ga", "roo"]
...
>>> f = Foo()
>>> dir(f)
['ga', 'kan', 'roo']

Note

Because dir() is supplied primarily as a convenience for use at an interactive prompt, it tries to supply an interesting set of names more than it tries to supply a rigorously or consistently defined set of names, and its detailed behavior may change across releases. For example, metaclass attributes are not in the result list when the argument is a class.

divmod(a, b)

Take two (non complex) numbers as arguments and return a pair of numbers consisting of their quotient and remainder when using long division. With mixed operand types, the rules for binary arithmetic operators apply. For plain and long integers, the result is the same as (a // b, a % b). For floating point numbers the result is (q, a % b), where q is usually math.floor(a / b) but may be 1 less than that. In any case q * b + a % b is very close to a, if a % b is non-zero it has the same sign as b, and 0 <= abs(a % b) < abs(b).

Changed in version 2.3: Using divmod() with complex numbers is deprecated.

enumerate(sequence[, start=0])

Return an enumerate object. sequence must be a sequence, an iterator, or some other object which supports iteration. The next() method of the iterator returned by enumerate() returns a tuple containing a count (from start which defaults to 0) and the corresponding value obtained from iterating over iterable. enumerate() is useful for obtaining an indexed series: (0, seq[0]), (1, seq[1]), (2, seq[2]), …. For example:

>>> for i, season in enumerate(['Spring', 'Summer', 'Fall', 'Winter']):
...     print i, season
0 Spring
1 Summer
2 Fall
3 Winter

New in version 2.3.

New in version 2.6: The start parameter.

eval(expression[, globals[, locals]])

The arguments are a string and optional globals and locals. If provided, globals must be a dictionary. If provided, locals can be any mapping object.

Changed in version 2.4: formerly locals was required to be a dictionary.

The expression argument is parsed and evaluated as a Python expression (technically speaking, a condition list) using the globals and locals dictionaries as global and local namespace. If the globals dictionary is present and lacks ‘__builtins__’, the current globals are copied into globals before expression is parsed. This means that expression normally has full access to the standard __builtin__ module and restricted environments are propagated. If the locals dictionary is omitted it defaults to the globals dictionary. If both dictionaries are omitted, the expression is executed in the environment where eval() is called. The return value is the result of the evaluated expression. Syntax errors are reported as exceptions. Example:

>>> x = 1
>>> print eval('x+1')
2

This function can also be used to execute arbitrary code objects (such as those created by compile()). In this case pass a code object instead of a string. If the code object has been compiled with 'exec' as the kind argument, eval()’s return value will be None.

Hints: dynamic execution of statements is supported by the exec statement. Execution of statements from a file is supported by the execfile() function. The globals() and locals() functions returns the current global and local dictionary, respectively, which may be useful to pass around for use by eval() or execfile().

execfile(filename[, globals[, locals]])

This function is similar to the exec statement, but parses a file instead of a string. It is different from the import statement in that it does not use the module administration — it reads the file unconditionally and does not create a new module. [#]_

The arguments are a file name and two optional dictionaries. The file is parsed and evaluated as a sequence of Python statements (similarly to a module) using the globals and locals dictionaries as global and local namespace. If provided, locals can be any mapping object.

Changed in version 2.4: formerly locals was required to be a dictionary.

If the locals dictionary is omitted it defaults to the globals dictionary. If both dictionaries are omitted, the expression is executed in the environment where execfile() is called. The return value is None.

Warning

The default locals act as described for function locals() below: modifications to the default locals dictionary should not be attempted. Pass an explicit locals dictionary if you need to see effects of the code on locals after function execfile() returns. execfile() cannot be used reliably to modify a function’s locals.

file(filename[, mode[, bufsize]])

Constructor function for the file type, described further in section bltin-file-objects. The constructor’s arguments are the same as those of the open() built-in function described below.

When opening a file, it’s preferable to use open() instead of invoking this constructor directly. file is more suited to type testing (for example, writing isinstance(f, file)).

New in version 2.2.

filter(function, iterable)

Construct a list from those elements of iterable for which function returns true. iterable may be either a sequence, a container which supports iteration, or an iterator. If iterable is a string or a tuple, the result also has that type; otherwise it is always a list. If function is None, the identity function is assumed, that is, all elements of iterable that are false are removed.

Note that filter(function, iterable) is equivalent to [item for item in iterable if function(item)] if function is not None and [item for item in iterable if item] if function is None.

See itertools.filterfalse() for the complementary function that returns elements of iterable for which function returns false.

getattr(object, name[, default])

Return the value of the named attributed of object. name must be a string. If the string is the name of one of the object’s attributes, the result is the value of that attribute. For example, getattr(x, 'foobar') is equivalent to x.foobar. If the named attribute does not exist, default is returned if provided, otherwise AttributeError is raised.

globals()

Return a dictionary representing the current global symbol table. This is always the dictionary of the current module (inside a function or method, this is the module where it is defined, not the module from which it is called).

hasattr(object, name)

The arguments are an object and a string. The result is True if the string is the name of one of the object’s attributes, False if not. (This is implemented by calling getattr(object, name) and seeing whether it raises an exception or not.)

hash(object)

Return the hash value of the object (if it has one). Hash values are integers. They are used to quickly compare dictionary keys during a dictionary lookup. Numeric values that compare equal have the same hash value (even if they are of different types, as is the case for 1 and 1.0).

help([object])

Invoke the built-in help system. (This function is intended for interactive use.) If no argument is given, the interactive help system starts on the interpreter console. If the argument is a string, then the string is looked up as the name of a module, function, class, method, keyword, or documentation topic, and a help page is printed on the console. If the argument is any other kind of object, a help page on the object is generated.

This function is added to the built-in namespace by the site module.

New in version 2.2.

hex(x)

Convert an integer number (of any size) to a hexadecimal string. The result is a valid Python expression.

Changed in version 2.4: Formerly only returned an unsigned literal.

id(object)

Return the “identity” of an object. This is an integer (or long integer) which is guaranteed to be unique and constant for this object during its lifetime. Two objects with non-overlapping lifetimes may have the same id() value. (Implementation note: this is the address of the object.)

input([prompt])

Equivalent to eval(raw_input(prompt)).

Warning

This function is not safe from user errors! It expects a valid Python expression as input; if the input is not syntactically valid, a SyntaxError will be raised. Other exceptions may be raised if there is an error during evaluation. (On the other hand, sometimes this is exactly what you need when writing a quick script for expert use.)

If the readline module was loaded, then input() will use it to provide elaborate line editing and history features.

Consider using the raw_input() function for general input from users.

isinstance(object, classinfo)

Return true if the object argument is an instance of the classinfo argument, or of a (direct or indirect) subclass thereof. Also return true if classinfo is a type object (new-style class) and object is an object of that type or of a (direct or indirect) subclass thereof. If object is not a class instance or an object of the given type, the function always returns false. If classinfo is neither a class object nor a type object, it may be a tuple of class or type objects, or may recursively contain other such tuples (other sequence types are not accepted). If classinfo is not a class, type, or tuple of classes, types, and such tuples, a TypeError exception is raised.

Changed in version 2.2: Support for a tuple of type information was added.

issubclass(class, classinfo)

Return true if class is a subclass (direct or indirect) of classinfo. A class is considered a subclass of itself. classinfo may be a tuple of class objects, in which case every entry in classinfo will be checked. In any other case, a TypeError exception is raised.

Changed in version 2.3: Support for a tuple of type information was added.

len(s)

Return the length (the number of items) of an object. The argument may be a sequence (string, tuple or list) or a mapping (dictionary).

locals()

Update and return a dictionary representing the current local symbol table.

Warning

The contents of this dictionary should not be modified; changes may not affect the values of local variables used by the interpreter.

Free variables are returned by locals() when it is called in a function block. Modifications of free variables may not affect the values used by the interpreter. Free variables are not returned in class blocks.

long([x[, radix]])

Convert a string or number to a long integer. If the argument is a string, it must contain a possibly signed number of arbitrary size, possibly embedded in whitespace. The radix argument is interpreted in the same way as for int(), and may only be given when x is a string. Otherwise, the argument may be a plain or long integer or a floating point number, and a long integer with the same value is returned. Conversion of floating point numbers to integers truncates (towards zero). If no arguments are given, returns 0L.

The long type is described in typesnumeric.

map(function, iterable, ...)

Apply function to every item of iterable and return a list of the results. If additional iterable arguments are passed, function must take that many arguments and is applied to the items from all iterables in parallel. If one iterable is shorter than another it is assumed to be extended with None items. If function is None, the identity function is assumed; if there are multiple arguments, map() returns a list consisting of tuples containing the corresponding items from all iterables (a kind of transpose operation). The iterable arguments may be a sequence or any iterable object; the result is always a list.

max(iterable[, args...][key])

With a single argument iterable, return the largest item of a non-empty iterable (such as a string, tuple or list). With more than one argument, return the largest of the arguments.

The optional key argument specifies a one-argument ordering function like that used for list.sort(). The key argument, if supplied, must be in keyword form (for example, max(a,b,c,key=func)).

Changed in version 2.5: Added support for the optional key argument.

min(iterable[, args...][key])

With a single argument iterable, return the smallest item of a non-empty iterable (such as a string, tuple or list). With more than one argument, return the smallest of the arguments.

The optional key argument specifies a one-argument ordering function like that used for list.sort(). The key argument, if supplied, must be in keyword form (for example, min(a,b,c,key=func)).

Changed in version 2.5: Added support for the optional key argument.

next(iterator[, default])

Retrieve the next item from the iterator by calling its next() method. If default is given, it is returned if the iterator is exhausted, otherwise StopIteration is raised.

New in version 2.6.

oct(x)

Convert an integer number (of any size) to an octal string. The result is a valid Python expression.

Changed in version 2.4: Formerly only returned an unsigned literal.

pow(x, y[, z])

Return x to the power y; if z is present, return x to the power y, modulo z (computed more efficiently than pow(x, y) % z). The two-argument form pow(x, y) is equivalent to using the power operator: x**y.

The arguments must have numeric types. With mixed operand types, the coercion rules for binary arithmetic operators apply. For int and long int operands, the result has the same type as the operands (after coercion) unless the second argument is negative; in that case, all arguments are converted to float and a float result is delivered. For example, 10**2 returns 100, but 10**-2 returns 0.01. (This last feature was added in Python 2.2. In Python 2.1 and before, if both arguments were of integer types and the second argument was negative, an exception was raised.) If the second argument is negative, the third argument must be omitted. If z is present, x and y must be of integer types, and y must be non-negative. (This restriction was added in Python 2.2. In Python 2.1 and before, floating 3-argument pow() returned platform-dependent results depending on floating-point rounding accidents.)

print([object, ...][, sep=' '][, end='n'][, file=sys.stdout])

Print object(s) to the stream file, separated by sep and followed by end. sep, end and file, if present, must be given as keyword arguments.

All non-keyword arguments are converted to strings like str() does and written to the stream, separated by sep and followed by end. Both sep and end must be strings; they can also be None, which means to use the default values. If no object is given, print() will just write end.

The file argument must be an object with a write(string) method; if it is not present or None, sys.stdout will be used.

Note

This function is not normally available as a builtin since the name print is recognized as the print statement. To disable the statement and use the print() function, use this future statement at the top of your module:

from __future__ import print_function

New in version 2.6.

property([fget[, fset[, fdel[, doc]]]])

Return a property attribute for new-style classes (classes that derive from object).

fget is a function for getting an attribute value, likewise fset is a function for setting, and fdel a function for del’ing, an attribute. Typical use is to define a managed attribute x:

class C(object):
    def __init__(self):
        self._x = None

    def getx(self):
        return self._x
    def setx(self, value):
        self._x = value
    def delx(self):
        del self._x
    x = property(getx, setx, delx, "I'm the 'x' property.")

If given, doc will be the docstring of the property attribute. Otherwise, the property will copy fget’s docstring (if it exists). This makes it possible to create read-only properties easily using property() as a decorator:

class Parrot(object):
    def __init__(self):
        self._voltage = 100000

    @property
    def voltage(self):
        """Get the current voltage."""
        return self._voltage

turns the voltage() method into a “getter” for a read-only attribute with the same name.

A property object has getter, setter, and deleter methods usable as decorators that create a copy of the property with the corresponding accessor function set to the decorated function. This is best explained with an example:

class C(object):
    def __init__(self):
        self._x = None

    @property
    def x(self):
        """I'm the 'x' property."""
        return self._x

    @x.setter
    def x(self, value):
        self._x = value

    @x.deleter
    def x(self):
        del self._x

This code is exactly equivalent to the first example. Be sure to give the additional functions the same name as the original property (x in this case.)

The returned property also has the attributes fget, fset, and fdel corresponding to the constructor arguments.

New in version 2.2.

Changed in version 2.5: Use fget’s docstring if no doc given.

Changed in version 2.6: The getter, setter, and deleter attributes were added.

raw_input([prompt])

If the prompt argument is present, it is written to standard output without a trailing newline. The function then reads a line from input, converts it to a string (stripping a trailing newline), and returns that. When EOF is read, EOFError is raised. Example:

>>> s = raw_input('--> ')
--> Monty Python's Flying Circus
>>> s
"Monty Python's Flying Circus"

If the readline module was loaded, then raw_input() will use it to provide elaborate line editing and history features.

reduce(function, iterable[, initializer])

Apply function of two arguments cumulatively to the items of iterable, from left to right, so as to reduce the iterable to a single value. For example, reduce(lambda x, y: x+y, [1, 2, 3, 4, 5]) calculates ((((1+2)+3)+4)+5). The left argument, x, is the accumulated value and the right argument, y, is the update value from the iterable. If the optional initializer is present, it is placed before the items of the iterable in the calculation, and serves as a default when the iterable is empty. If initializer is not given and iterable contains only one item, the first item is returned.

reload(module)

Reload a previously imported module. The argument must be a module object, so it must have been successfully imported before. This is useful if you have edited the module source file using an external editor and want to try out the new version without leaving the Python interpreter. The return value is the module object (the same as the module argument).

When reload(module) is executed:

• Python modules’ code is recompiled and the module-level code reexecuted, defining a new set of objects which are bound to names in the module’s dictionary. The init function of extension modules is not called a second time.
• As with all other objects in Python the old objects are only reclaimed after their reference counts drop to zero.
• The names in the module namespace are updated to point to any new or changed objects.
• Other references to the old objects (such as names external to the module) are not rebound to refer to the new objects and must be updated in each namespace where they occur if that is desired.

There are a number of other caveats:

If a module is syntactically correct but its initialization fails, the first import statement for it does not bind its name locally, but does store a (partially initialized) module object in sys.modules. To reload the module you must first import it again (this will bind the name to the partially initialized module object) before you can reload() it.

When a module is reloaded, its dictionary (containing the module’s global variables) is retained. Redefinitions of names will override the old definitions, so this is generally not a problem. If the new version of a module does not define a name that was defined by the old version, the old definition remains. This feature can be used to the module’s advantage if it maintains a global table or cache of objects — with a try statement it can test for the table’s presence and skip its initialization if desired:

try:
    cache
except NameError:
    cache = {}

It is legal though generally not very useful to reload built-in or dynamically loaded modules, except for sys, __main__ and __builtin__. In many cases, however, extension modules are not designed to be initialized more than once, and may fail in arbitrary ways when reloaded.

If a module imports objects from another module using from … import …, calling reload() for the other module does not redefine the objects imported from it — one way around this is to re-execute the from statement, another is to use import and qualified names (module.*name*) instead.

If a module instantiates instances of a class, reloading the module that defines the class does not affect the method definitions of the instances — they continue to use the old class definition. The same is true for derived classes.

repr(object)

Return a string containing a printable representation of an object. This is the same value yielded by conversions (reverse quotes). It is sometimes useful to be able to access this operation as an ordinary function. For many types, this function makes an attempt to return a string that would yield an object with the same value when passed to eval(), otherwise the representation is a string enclosed in angle brackets that contains the name of the type of the object together with additional information often including the name and address of the object. A class can control what this function returns for its instances by defining a __repr__() method.

reversed(seq)

Return a reverse iterator. seq must be an object which has a __reversed__() method or supports the sequence protocol (the __len__() method and the __getitem__() method with integer arguments starting at 0).

New in version 2.4.

Changed in version 2.6: Added the possibility to write a custom __reversed__() method.

round(x[, n])

Return the floating point value x rounded to n digits after the decimal point. If n is omitted, it defaults to zero. The result is a floating point number. Values are rounded to the closest multiple of 10 to the power minus n; if two multiples are equally close, rounding is done away from 0 (so. for example, round(0.5) is 1.0 and round(-0.5) is -1.0).

setattr(object, name, value)

This is the counterpart of getattr(). The arguments are an object, a string and an arbitrary value. The string may name an existing attribute or a new attribute. The function assigns the value to the attribute, provided the object allows it. For example, setattr(x, 'foobar', 123) is equivalent to x.foobar = 123.

slice([start, ]stop[, step])

Return a slice object representing the set of indices specified by range(start, stop, step). The start and step arguments default to None. Slice objects have read-only data attributes start, stop and step which merely return the argument values (or their default). They have no other explicit functionality; however they are used by Numerical Python and other third party extensions. Slice objects are also generated when extended indexing syntax is used. For example: a[start:stop:step] or a[start:stop, i]. See itertools.islice() for an alternate version that returns an iterator.

sorted(iterable[, cmp[, key[, reverse]]])

Return a new sorted list from the items in iterable.

The optional arguments cmp, key, and reverse have the same meaning as those for the list.sort() method (described in section typesseq-mutable).

cmp specifies a custom comparison function of two arguments (iterable elements) which should return a negative, zero or positive number depending on whether the first argument is considered smaller than, equal to, or larger than the second argument: cmp=lambda x,y: cmp(x.lower(), y.lower()). The default value is None.

key specifies a function of one argument that is used to extract a comparison key from each list element: key=str.lower. The default value is None.

reverse is a boolean value. If set to True, then the list elements are sorted as if each comparison were reversed.

In general, the key and reverse conversion processes are much faster than specifying an equivalent cmp function. This is because cmp is called multiple times for each list element while key and reverse touch each element only once. To convert an old-style cmp function to a key function, see the CmpToKey recipe in the ASPN cookbook.

New in version 2.4.

staticmethod(function)

Return a static method for function.

A static method does not receive an implicit first argument. To declare a static method, use this idiom:

class C:
    @staticmethod
    def f(arg1, arg2, ...): ...

The @staticmethod form is a function decorator – see the description of function definitions in function for details.

It can be called either on the class (such as C.f()) or on an instance (such as C().f()). The instance is ignored except for its class.

Static methods in Python are similar to those found in Java or C++. For a more advanced concept, see classmethod() in this section.

For more information on static methods, consult the documentation on the standard type hierarchy in types.

New in version 2.2.

Changed in version 2.4: Function decorator syntax added.

sum(iterable[, start])

Sums start and the items of an iterable from left to right and returns the total. start defaults to 0. The iterable’s items are normally numbers, and are not allowed to be strings. The fast, correct way to concatenate a sequence of strings is by calling ''.join(sequence). Note that sum(range(n), m) is equivalent to reduce(operator.add, range(n), m) To add floating point values with extended precision, see math.fsum().

New in version 2.3.

super(type[, object-or-type])

Return a proxy object that delegates method calls to a parent or sibling class of type. This is useful for accessing inherited methods that have been overridden in a class. The search order is same as that used by getattr() except that the type itself is skipped.

The __mro__ attribute of the type lists the method resolution search order used by both getattr() and super(). The attribute is dynamic and can change whenever the inheritance hierarchy is updated.

If the second argument is omitted, the super object returned is unbound. If the second argument is an object, isinstance(obj, type) must be true. If the second argument is a type, issubclass(type2, type) must be true (this is useful for classmethods).

Note

super() only works for new-style classes.

There are two typical use cases for super. In a class hierarchy with single inheritance, super can be used to refer to parent classes without naming them explicitly, thus making the code more maintainable. This use closely parallels the use of super in other programming languages.

The second use case is to support cooperative multiple inheritance in a dynamic execution environment. This use case is unique to Python and is not found in statically compiled languages or languages that only support single inheritance. This makes it possible to implement “diamond diagrams” where multiple base classes implement the same method. Good design dictates that this method have the same calling signature in every case (because the order of calls is determined at runtime, because that order adapts to changes in the class hierarchy, and because that order can include sibling classes that are unknown prior to runtime).

For both use cases, a typical superclass call looks like this:

class C(B):
    def method(self, arg):
        super(C, self).method(arg)

Note that super() is implemented as part of the binding process for explicit dotted attribute lookups such as super().__getitem__(name). It does so by implementing its own __getattribute__() method for searching classes in a predictable order that supports cooperative multiple inheritance. Accordingly, super() is undefined for implicit lookups using statements or operators such as super()[name].

Also note that super() is not limited to use inside methods. The two argument form specifies the arguments exactly and makes the appropriate references.

New in version 2.2.

type(object)

Return the type of an object. The return value is a type object. The isinstance() built-in function is recommended for testing the type of an object.

With three arguments, type() functions as a constructor as detailed below.

vars([object])

Without arguments, return a dictionary corresponding to the current local symbol table. With a module, class or class instance object as argument (or anything else that has a __dict__ attribute), returns a dictionary corresponding to the object’s symbol table.

Warning

The returned dictionary should not be modified: the effects on the corresponding symbol table are undefined. [#]_

xrange([start, ]stop[, step])

This function is very similar to range(), but returns an “xrange object” instead of a list. This is an opaque sequence type which yields the same values as the corresponding list, without actually storing them all simultaneously. The advantage of xrange() over range() is minimal (since xrange() still has to create the values when asked for them) except when a very large range is used on a memory-starved machine or when all of the range’s elements are never used (such as when the loop is usually terminated with break).

Note

xrange() is intended to be simple and fast. Implementations may impose restrictions to achieve this. The C implementation of Python restricts all arguments to native C longs (“short” Python integers), and also requires that the number of elements fit in a native C long. If a larger range is needed, an alternate version can be crafted using the itertools module: islice(count(start, step), (stop-start+step-1)//step).

zip([iterable, ...])

This function returns a list of tuples, where the i-th tuple contains the i-th element from each of the argument sequences or iterables. The returned list is truncated in length to the length of the shortest argument sequence. When there are multiple arguments which are all of the same length, zip() is similar to map() with an initial argument of None. With a single sequence argument, it returns a list of 1-tuples. With no arguments, it returns an empty list.

The left-to-right evaluation order of the iterables is guaranteed. This makes possible an idiom for clustering a data series into n-length groups using zip(*[iter(s)]*n).

zip() in conjunction with the * operator can be used to unzip a list:

>>> x = [1, 2, 3]
>>> y = [4, 5, 6]
>>> zipped = zip(x, y)
>>> zipped
[(1, 4), (2, 5), (3, 6)]
>>> x2, y2 = zip(*zipped)
>>> x == x2, y == y2
True

New in version 2.0.

Changed in version 2.4: Formerly, zip() required at least one argument and zip() raised a TypeError instead of returning an empty list.

__import__(name[, globals[, locals[, fromlist[, level]]]])

Note

This is an advanced function that is not needed in everyday Python programming.

This function is invoked by the import statement. It can be replaced (by importing the builtins module and assigning to builtins.__import__) in order to change semantics of the import statement, but nowadays it is usually simpler to use import hooks (see PEP 302). Direct use of __import__() is rare, except in cases where you want to import a module whose name is only known at runtime.

The function imports the module name, potentially using the given globals and locals to determine how to interpret the name in a package context. The fromlist gives the names of objects or submodules that should be imported from the module given by name. The standard implementation does not use its locals argument at all, and uses its globals only to determine the package context of the import statement.

level specifies whether to use absolute or relative imports. The default is -1 which indicates both absolute and relative imports will be attempted. 0 means only perform absolute imports. Positive values for level indicate the number of parent directories to search relative to the directory of the module calling __import__().

When the name variable is of the form package.module, normally, the top-level package (the name up till the first dot) is returned, not the module named by name. However, when a non-empty fromlist argument is given, the module named by name is returned.

For example, the statement import spam results in bytecode resembling the following code:

spam = __import__('spam', globals(), locals(), [], -1)

The statement import spam.ham results in this call:

spam = __import__('spam.ham', globals(), locals(), [], -1)

Note how __import__() returns the toplevel module here because this is the object that is bound to a name by the import statement.

On the other hand, the statement from spam.ham import eggs, sausage as saus results in

_temp = __import__('spam.ham', globals(), locals(), ['eggs', 'sausage'], -1)
eggs = _temp.eggs
saus = _temp.sausage

Here, the spam.ham module is returned from __import__(). From this object, the names to import are retrieved and assigned to their respective names.

If you simply want to import a module (potentially within a package) by name, you can get it from sys.modules:

>>> import sys
>>> name = 'foo.bar.baz'
>>> __import__(name)
<module 'foo' from ...>
>>> baz = sys.modules[name]
>>> baz
<module 'foo.bar.baz' from ...>

Changed in version 2.5: The level parameter was added.

Changed in version 2.5: Keyword support for parameters was added.

XX JJ: Maybe this can be a separate chapter?

XXX let’s just pull in the actual documentation, then modify/augment as desired. I still prefer the grouping that we are doing here, especially if we can create an index.

XXX Let’s list these by functionality, that is Group by functionality; this is the standard docs, augmented by our perspectives on how to use them.

Alternative Ways to Define Functions

The def keyword is not the only way to define a function. Here are some alternatives:

• lambda functions. The lambda keyword creates an unnamed function. Some people like this because it requires minimal space, especially when used in a callback:

  XXX lambda in a keyed sort, maybe combine last name, first name?

XX JJ: Example of using a lambda function to combine two strings. In this case, a first

and last name

 >>> name_combo = lambda first,last: first + ' ' + last
 >>> name_combo('Jim','Baker')
 'Jim Baker'

  XXX gen exp ex

* Classes. In addition, we can also create objects with classes
  whose instance objects look like ordinary functions.  Objects
  supporting the __call__ protocol. This should be covered in a
  later chapter.  For Java developers, this is familiar. Classes
  implement such single-method interfaces as Callable or Runnable.

* Bound methods. Instead of calling x.a(), I can pass x.a as a
  parameter or bind to another name. Then I can invoke this
  name. The first parameter of the method will be passed the bound
  object, which in OO terms is the receiver of the method. This is
  a simple way of creating callbacks. (In Java you would have just
  passed the object of course, then having the callback invoke the
  appropriate method such as `call` or `run`.)

* staticmethod, classmethod, descriptors functools, such as for
  partial construction.

* Other function constructors, including yours?

Lambda Functions

Limitations.

XX JJ: I imagine that this will be filled out, should the lambda function from

above be moved into this section?

Generator Functions

Generators are functions that construct objects implementing Python’s iterator protocol.

iter() - obj.__iter__ Call obj.next

Advance to the next point by calling the special method next. Usually that’s done implicitly, typically through a loop or a consuming function that accepts iterators, including generators.

XX JJ: More explanation needed here. It would be a good idea to mention

that StopIteration is thrown when a generator has been “used up” if the generator is not being used in a loop.

A generator function is written so that it consists of one or more yield points, which are marked through the use of the keyword yield:

def g():
    print "before yield point 1"
    yield 1
    print "after 1, before 2"
    yield 2
    yield 3

XX JJ: Perhaps anotherp more useful example:

>>> def step_to(factor, stop):
...     step = factor
...     start = 0
...     while start <= stop:
...         yield start
...         start += step
...
>>> for x in step_to(1, 10):
...     print x
...
0
1
2
3
4
5
6
7
8
9
10
>>> for x in step_to(2, 10):
...     print x
...
0
2
4
6
8
10
>>>

If the yield keyword is seen in the scope of a function, it’s compiled as if it’s a generator function.

Unlike other functions, you use the return statement only to say, “I’m done”, that is, to exit the generator:

XXX code

XX JJ: Let’s change the step_to function just a bit to check and ensure

that the factor is less than the stopping point. We’ll add a return statement to exit the generator if the factor is gt or equal to the stop.

>>> def step_return(factor, stop):
...     step = factor
...     start = 0
...     if factor >= stop:
...         return
...     while start <= stop:
...         yield start
...         start += step
...
>>> for x in step_return(1,10):
...     print x
...
0
1
2
3
4
5
6
7
8
9
10
>>> for x in step_return(3,10):
...     print x
...
0
3
6
9
>>> for x in step_return(3,3):
...     print x
...

You can’t return an argument:

def g():
    yield 1
    yield 2
    return None

for i in g():
    print i

SyntaxError: 'return' with argument inside generator

But it’s not necessary to explicitly return. You can think of return as acting like a break in a for-loop or while-loop.

Many useful generators actually will have an infinite loop around their yield expression, instead of ever exiting, explicitly or not:

XXX while True:
   yield stuff

This works because a generator object can be garbage collected, just like any other object implementing the iteration protocol. The fact that it uses the machinery of function objects to implement itself doesn’t matter.

How it actually works

Generators are actually compiled differently from other functions. Each yield point saves the state of unnamed local variables (Java temporaries) into the frame object, then returns the value to the function that had called next (or send in the case of a coroutine). The generator is then indefinitely suspended, just like any other iterator. Upon calling next again, the generator is resumed by restoring these local variables, then executing the next bytecode instruction following the yield point. This process continues until the generator is either garbage collected or it exits.

You can determine if the underlying function is a generator if its code object has the CO_GENERATOR flag set in co_flags.

Generators can also be resumed from any thread, although some care is necessary to ensure that underlying system state is shared (or compatible). We will explore how to use effectively use this capability in the chapter on concurrency.

This is an alternative way to create the generator object. Please note this is not a generator function! It’s the equivalent to what a generator function returns when called.

XX JJ: Maybe we should say “yields” when called?

. Creates an unnamed generator. But cover this later with respect to generators. Note that generators are not callable objects:

>>> x = (2 * x for x in [1,2,3,4])
>>> x
<generator object at 0x1>
>>> x()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'generator' object is not callable

XX JJ: Show call to x.next()

>>> for v in x:
...     print v
...
2
4
6
8
>>>

Using Generators

Python iteration protocol. iter, next.

Generator Example

contextlib

Jar scanner

How to use in interesting ways with Java. For example, we wrap everything in Java that supports java.util.Iterator so it supports the Python iteration protocol.

Maybe something simple like walking a directory tree?

XX JJ: I like the walker idea. Perhaps a different implementation of Frank’s from chapter 7?

In conjunction with glob type functionality? And possibly other analysis. Maybe process every single file, etc. That could be sort of cool, and something I don’t think is so easy from Java (no, it’s not). Also we will want to wrap it up with RAII semantics too, to ensure closing.

Lastly - what sort of Java client code would want such an iterator? That’s the other part of the equation to be solved here. Maybe some sort of plugin? Don’t want to make the example too contrived. Some relevant discussion here in a Java tutorial: http://java.sun.com/docs/books/tutorial/essential/io/walk.html

What about a simple Jar scanner? That’s sort of handy… and feeds into other functionality too. Could be the subject of Ant integration too. (Or Maven or Ivy, but perhaps this is going beyond my knowledge here.)

One common usage of a generator is to watch a log file for changes (tail -f). We can create something similar with the NIO package, although this does require the use of a thread for the watcher (but this of course can be multiplexed across multiple directories).

Watching a directory for changes. In CPython, this requires fcntl on Unix/Linux systems, and the use of a completely different Win32 API on Windows systems. http://stackoverflow.com/questions/182197/how-do-i-watch-a-file-for-changes-using-python Java provides a simple approach: http://java.sun.com/docs/books/tutorial/essential/io/notification.html - how to do it in Java

Generator Expressions

XXX Maybe something simple with Java Mail? Could show how to attach files that meet a certain criteria?

Namespaces, Nested Scopes and Closures

Functions can be nested.

Most importantly this allows the construction of closures.

Namespaces Note that you can introduce other namespaces into your function definition. So:

def f():
    from NS import A, B

Function Decorators

Function decorators are two things:

• A convenient syntax that describes how to transform a function. You might want to memoize a given function, so it uses a cache, with a desired policy, to remember a result for a given set of parameters. Or you may want to create a static method in a class.
• A powerful, yet simple design where the decorator is a function on function that results in the decorated, or transformed, function.

(Class decorators are similar, except they are functions on classes).

XXX example - XXX How about a decorator for Java integration? eg add support of a given interface to facilitate callbacks

Creating Decorators

Memoization decorator. For our same Fibonacci example.

Often a function definition is not the simplest way to write the desired decorator function. Instead, you might want to create a class, as we described in alternate ways to create function objects.

XXX In addition, functools, specifically the wraps function.

XXX ref Eckel’s article on decorators.

Using Decorators

XXX Chopping block

Coroutines

One thing

to remember: coroutines do not mix with generators, despite being related in both syntax and implementation. Coroutines use push; generators use pull.

XXX The PyCon tutorial on coroutines has some useful coroutine examples - certainly need similar coverage.

XXX Might be nice to show how to use this in conjunction with parallelism. but that’s a later chapter anyway

XX JJ: I think that Coroutines need to be in this book in some form…don’t cut them out. Even

if it is just a short section with an example.

Advanced Function Usage

Frames Tracebacks Profiling and tracing Introspection on functions - various attributes, etc, not to mention the use of inspect

Function Code Bodies

Jython, like CPython, only has one unit of compilation, the function code body. When a module is compiled, every function in it is compiled even if it’s not ultimately bound to a name. In addition, a script or module is itself treated as a function when compiled. These function definitions are compiled to Java bytecode. (There’s experimental support for other formats, namely Python bytecode, which we may see be used in later versions of Jython.)

Partitioning this global namespace with shadowing.

The inspect module. Determining parameters, etc. One thing that is not supported: introspecting on code objects themselves.

Special syntax support for operators

x.a del x x[i] etc.

All operators are also available as functions from the :module:`operator`. It should be noted that operators on built-in types (int, str, dict, etc.) will usually execute faster on the JVM because they do not require dynamic dispatch. Invokedynamic, part of JDK 7, is exciting because it makes that cost go away, but we will have to wait for that.