The code files for this chapter can be found at https://git.io/fj2UB.

Many of the examples in the book will center on simulations of a process with a number of moderately complex state changes. The card game of Blackjack involves a few rules and a few state changes during play. If you're unfamiliar with the game of Blackjack, here's an overview.

The objective of the game is to accept cards from the dealer to create a hand that has a point total that is between the dealer's total and twenty-one. The dealer's hand is only partially revealed, forcing the player to make a decision without knowing the dealer's total or the subsequent cards from the deck.

The number cards (2 to 10) have point values equal to the number. The face cards (Jack, Queen, and King) are worth 10 points. The Ace is worth either eleven points or one point. When using an ace as eleven points, the value of the hand is soft. When using an ace as one point, the value is hard.

A hand with an Ace and a seven, therefore, has a hard total of eight and a soft total of 18. This leads the player to choose to take extra cards. If the dealer is showing a face card, it's very likely the dealer is holding twenty points, and the player may not want to risk taking another card.

Each suit has four two-card combinations that total 21. These are all called Blackjack, even though only one of the four combinations involves a Jack. These combinations often provide a bonus payout, because there are only four of them available.

Most of the game is about proper choice of cards. There is, of course, a betting element. The distinction between playing and betting is made somewhat more complicated by the provision to split one hand into two hands. This is allowed when the player's two cards have the same rank. This option will be detailed in the next section on how the game is played.

The mechanics of play generally work as follows. The details can vary, but the outline is similar:

• First, the player and dealer each get two cards. The player, of course, knows the value of both of their cards. They're dealt face up in a casino.
• One of the dealer's cards is revealed to the player. It's displayed face up. The player, therefore, knows a little bit about the dealer's hand, but not everything. This is typical of more complex simulations where partial information is available and statistical modeling is required to make appropriate decisions.
• If the dealer has an Ace showing, the player is offered the opportunity to place an additional insurance bet. This is a special case, and is typical of more complex simulations where there are exceptions.
• For the balance of the game, the player can elect to receive cards, or stop receiving cards. There are four choices available:
  • The player can hit, which means take another card.
  • They player can or stand or stand pat with the cards dealt.
  • If the player's cards match, the hand can be split. This entails an additional bet, and the two hands are played separately. 
  • The player can double their bet before taking one last card. This is called doubling down.

The final evaluation of the hand works as follows:

• If the player went over 21, the hand is a bust, the player loses, and the dealer's face-down card is irrelevant. This provides an advantage to the dealer.
• If the player's total is 21 or under, then the dealer takes cards according to a simple, fixed rule. The dealer must hit a hand that totals less than 18; the dealer must stand on a hand that totals 18 or more.
• If the dealer goes bust, the player wins.
• If both the dealer and player are 21 or under, the hands are compared. The higher total is the winner. In the event of a tie, the game is a push, neither a win nor a loss. If the player wins with 21, they win a larger payout, usually 1.5 times the bet.

The rules can vary quite a bit. We'll elide these details to focus on the Python code required for simulation.

In the case of blackjack, there are actually two kinds of strategies that the player must use:

• A strategy for deciding what play to make: take insurance, hit, stand, split, or double down.
• A strategy for deciding what amount to bet. A common statistical fallacy leads players to raise and lower their bets in an attempt to preserve their winnings and minimize their losses. These are interesting, stateful algorithms in spite of the underlying fallacies.

These two sets of strategies are, of course, prime examples of the Strategy design pattern.

We'll use elements of the game, such as the player, hand, and card, as examples for object modeling. We won't design the entire simulation. We'll focus on elements of this game because they have some nuance, but aren't terribly complex.

The cards are relatively simple, immutable objects. There are a variety of modeling techniques available. Cards fall into a simple class hierarchy of the number cards, face cards, and the Ace.  There are simple containers, including hands of card instances, and decks of cards as well. These are stateful collections with cards being added and removed. There are a number of ways to implement this in Python and we'll look at many alternatives. We also need to look at the player as a whole. A player will have a sequence of hands, as well as a betting strategy and a Blackjack play strategy. This is a rather complex composite object.

One of the essential concepts for mastering object-oriented Python is to understand how object methods are implemented. Let's look at a relatively simple Python interaction:

>>> f = [1, 1, 2, 3]
>>> f += [f[-1] + f[-2]]
>>> f
[1, 1, 2, 3, 5]

We've created a list, f, with a sequence of values. We then mutated this list using the += operator to append a new value. The f[-1] + f[-2] expression computes the new value to be appended.

The value of f[-1] is implemented using the list object's __getitem__() method. This is a core pattern of Python: the simple operator-like syntax is implemented by special methods. The special methods have names surrounded with __ to make them distinctive. For simple prefix and suffix syntax, the object is obvious; f[-1] is implemented as f.__getitem__(-1).

The additional operation is similarly implemented by the __add__() special method. In the case of a binary operator, Python will try both operands to see which one offers the special method. In this example, both operands are integers, and both will provide a suitable implementation. In the case of mixed types, the implementation of the binary operator may coerce one value into another type. f[-1] + f[-2], then, is implemented as f.__getitem__(-1).__add__(f.__getitem__(-2)).

The update of f by the += operator is implemented by the __iadd__() special method. Consequently, f += [x] is implemented as f.__iadd__([x]).

Throughout the first eight chapters, we'll look very closely at these special methods and how we can design our classes to integrate very tightly with Python's built-in language features. Mastering the special methods is the essence of mastering object-oriented Python.

Python is often described as Batteries Included programming. Everything required is available directly as part of a single download. This provides the runtime, the standard library, and the IDLE editor as a simple development environment.

It's very easy to download and install Python 3.7 and start running it interactively on the desktop. The example in the previous section included the >>> prompt from interactive Python. 

If you're using the Iron Python (IPython) implementation, the interaction will look like this:

In [1]: f = [1, 1, 2, 3]
In [3]: f += [f[-1] + f[-2]]
In [4]: f
Out[4]: [1, 1, 2, 3, 5]

The prompt is slightly different, but the language is the same. Each statement is evaluated as it is presented to Python. 

This is handy for some experimentation. Our goal is to build tools, frameworks, and applications. While many of the examples will be shown in an interactive style, most of the actual programming will be via script files.

Running examples interactively makes a profound statement. Well-written Python code should be simple enough that it can be run from the command line.

Interactive use is not our goal. Exercising code from the >>> prompt is a quality test for complexity. If the code is too complex to exercise it from the >>> prompt, then refactoring is needed.

The focus of this book is on creating complete scripts, modules, packages, and applications. Even though some examples are shown in interactive mode, the objective is to create Python files. These files may be as simple as a script or as complex as a directory with files to create a web application.

Tools such as mypy, pytest, and pylint work with Python files. Preparing script files can be done with almost any text editor. It's best, however, to work with an IDE, where a number of tools can be provided to help develop applications and scripts.

A common question is, "What is the best IDE for doing Python development?" The short answer to this question is that the IDE choice doesn't matter very much. The number of development environments that support Python is vast and they are all very easy to use. The long answer requires a conversation about what attributes would rank an IDE as being the best.

The Spyder IDE is part of the Anaconda distribution. This makes it readily accessible to developers who've downloaded Anaconda. The IDLE editor is part of the Python distribution, and provides a simple environment for using Python and building scripts. PyCharm has a commercial license as well as a community edition, it provides a large number of features, and was used to prepare all the examples in this book.

The author makes use of having both an editor, an integrated Python prompt, and unit test results all readily available. PyCharm works well with the conda environments, avoiding confusion over what packages are installed. 

A search on the internet will provide a long list of other tools. See the IDE Python wiki page for numerous alternatives (https://wiki.python.org/moin/IntegratedDevelopmentEnvironments).

All of the examples in the book were prepared using the black tool to provide consistent formatting. Some additional manual adjustments were made to keep code within the narrow sizes of printed material.

A common alternative to using black is to use pylint to identify formatting problems. These can then be corrected. In addition to detailed analysis of code quality, the pylint tool offers a numeric quality score. For this book, some pylint rules needed to be disabled. For example, the modules often have imports that are not in the preferred order; some modules also have imports that are relevant to doctest examples, and appear to be unused; some examples use global variables; and some class definitions are mere skeletons without appropriate method definitions. 

Using pylint to locate potential problems is essential. It's often helpful to silence pylint warnings. In the following example, we need to silence a pylint warning about the test_list variable name being invalid as a global variable:

# pylint: disable=invalid-name
test_list = """
   >>> f = [1, 1, 2, 3]
    >>> f += [f[-1] + f[-2]]
    >>> f
    [1, 1, 2, 3, 5]
   """

if __name__ == "__main__":
    import doctest
    __test__ = {name: value 
        for name, value in locals().items() 
            if name.startswith("test_")}
    doctest.testmod(verbose=False)

Besides helping enforce a consistent style, the pylint warnings are helpful for identifying spelling mistakes and a list of common errors. For example, the instance variable is commonly self. An accidental spelling error of sefl will be found by pylint. 

Python 3 permits the use of type hints. The hints are present in assignment statements, function, and class definitions. They're not used directly by Python when the program runs. Instead, they're used by external tools to examine the code for improper use of types, variables, and functions. Here's a simple function with type hints:

def F(n: int) -> int:
    if n in (0, 1):
        return 1
    else:
        return F(n-1) + F(n-2)

print("Good Use", F(8))
print("Bad Use", F(355/113))

When we run the mypy program, we'll see an error such as the following:

Chapter_1/ch01_ex3.py:23: error: Argument 1 to "F" has incompatible type "float"; expected "int"

This message informs us of the location of the error: the file is Chapter_1/ch01_ex3.py, which is the 23^rd line of the file. The details tell us that the function, F, has an improper argument value. This kind of problem can be difficult to see. In some cases, unit tests might not cover this case very well, and it's possible for a program to harbor subtle bugs because data of an improper type might be used.

We'll make use of the timeit module to compare the actual performance of different object-oriented designs and Python constructs. We'll focus on the timeit() function in this module. This function creates a Timer object that's used to measure the execution of a given block of code. We can also provide some preparatory code that creates an environment. The return value from this function is the time required to run the given block of code.

The default count is 100,000. This provides a meaningful time that averages out other OS-level activity on the computer doing the measurement. For complex or long-running statements, a lower count may be prudent.

Here's a simple interaction with timeit:

>>> timeit.timeit("obj.method()", 
... """
... class SomeClass:
...     def method(self):
...         pass
... obj= SomeClass()
... """)
0.1980541350058047

The code to be measured is obj.method(). It is provided to timeit() as a string. The setup code block is the class definition and object construction. This code block, too, is provided as a string. It's important to note that everything required by the statement must be in the setup. This includes all imports, as well as all variable definitions and object creation.

This example showed that 100,000 method calls that do nothing costs 0.198 seconds.

Unit testing is absolutely essential.

If there's no automated test to show a particular element functionality, then the feature doesn't really exist. Put another way, it's not done until there's a test that shows that it's done.

We'll touch, tangentially, on testing. If we delved into testing each object-oriented design feature, the book would be twice as long as it is. Omitting the details of testing has the disadvantage of making good unit tests seem optional. They're emphatically not optional.

Python offers two built-in testing frameworks. Most applications and libraries will make use of both. One general wrapper for testing is the unittest module. In addition, many public API docstrings will have examples that can be found and used by the doctest module. Also, unittest can incorporate doctest.

The pytest tool can locate test cases and execute them. This is a very useful tool, but must be installed separately from the rest of Python.

One lofty ideal is that every class and function has at least a unit test. The important, visible classes and functions will often also have doctest. There are other lofty ideals: 100% code coverage; 100% logic path coverage, and so on.

Pragmatically, some classes don't need testing. A class that extends typing.NamedTuple, for example, doesn't really need a sophisticated unit test. It's important to test the unique features of a class you've written and not the features inherited from the standard library.

Generally, we want to develop the test cases first, and then write code that fits the test cases. The test cases formalize the API for the code. This book will reveal numerous ways to write code that has the same interface. Once we've defined an interface, there are still numerous candidate implementations that fit the interface. One set of tests will apply to several different object-oriented designs.

One general approach to using the unittest and pytest tools is to create at least three parallel directories for your project:

• myproject: This directory is the final package that will be installed in lib/site-packages for your package or application. It has an __init__.py file. We'll put our files in here for each module.
• tests: This directory has the test scripts. In some cases, the scripts will parallel the modules. In some cases, the scripts may be larger and more complex than the modules themselves.
• docs: This has other documentation. We'll touch on this in the next section, as well as a chapter in part three.

In some cases, we'll want to run the same test suite on multiple candidate classes so that we can be sure each candidate works. There's no point in doing timeit comparisons on code that doesn't actually work.

All Python code should have docstrings at the module, class and method level. Not every single method requires a docstring. Some method names are really well chosen, and little more needs to be said. Most times, however, documentation is essential for clarity.

Python documentation is often written using the reStructuredText (RST) markup.

Throughout the code examples in the book, however, we'll omit docstrings. The omission keeps the book to a reasonable size. This gap has the disadvantage of making docstrings seem optional. They're emphatically not optional.

The docstring material is used three ways by Python:

• The internal help() function displays the docstrings.
• The doctest tool can find examples in docstrings and run them as test cases.
• External tools, such as sphinx and pydoc, can produce elegant documentation extracts from these strings.

Because of the relative simplicity of RST, it's quite easy to write good docstrings. We'll look at documentation and the expected markup in detail in Chapter 18, Coping with the Command Line. For now, however, we'll provide a quick example of what a docstring might look like:

def factorial(n: int) -> int:
    """
    Compute n! recursively.

    :param n: an integer >= 0
    :returns: n!

    Because of Python's stack limitation, this won't compute a value larger than about 1000!.

    >>> factorial(5)
    120
    """
    if n == 0:
        return 1
    return n*factorial(n-1)

This shows the RST markup for the n parameter and the return value. It includes an additional note about limitations. It also includes a doctest example that can be used to validate the implementation using the doctest tool. The use of :param n: and :return: identifies text that will be used by the sphinx tool to provide proper formatting and indexing of the information.

Most of the tools required must be added to the Python 3.7 environment. There are two approaches in common use:

• Use pip to install everything.
• Use conda to create an environment. Most of the tools described in this book are part of the Anaconda distribution.

The pip installation uses a single command:

python3 -m pip install pyyaml sqlalchemy jinja2 pytest sphinx mypy pylint black

This will install all of the required packages and tools in your current Python environment.

The conda installation creates a conda environment to keep the book's material separate from any other projects:

1. Install conda. If you have already installed Anaconda, you have the Conda tool, nothing more needs to be done. If you don't have Anaconda yet, then install miniconda, which is the ideal way to get started. Visit https://conda.io/miniconda.html and download the appropriate version of conda for your platform.
2. Use conda to build and activate the new environment.

3. Then upgrade pip. This is needed because the default pip installation in the Python 3.7 environment is often slightly out of date.
4. Finally, install black. This is required because black is not currently in any of the conda distribution channels.

Here are the commands:

$ conda create --name mastering python=3.7 pyyaml sqlalchemy jinja2 
  pytest sphinx mypy pylint
$ conda activate mastering
$ python3 -m pip install --upgrade pip
$ python3 -m pip install black

The suite of tools (pytest, sphinx, mypy, pylint, and black) are essential for creating high-quality, reliable Python programs. The other components, pyyaml, sqlalchemy, and jinja2, are helpful for building useful applications.

In this chapter, we've surveyed the game of Blackjack. The rules have a moderate level of complexity, providing a framework for creating a simulation. Simulation was one of the first uses for OOP and remains a rich source of programming problems that illustrate language and library strengths.

This chapter introduces the way the Python runtime uses special methods to implement the various operators. The bulk of this book will show ways to make use of the special methods names for creating objects that interact seamlessly with other Python features.

We've also looked at a number of tools that will be required to build good Python applications. This includes the IDE, the mypy program for checking type hints, and the black and pylint programs for getting to a consistent style. We also looked at the timeit, unittest, and doctest modules for doing essential performance and functional testing. For final documentation of a project, it's helpful to install sphinx. The installation of these extra components can be done with pip or conda. The pip tool is part of Python, the conda tool requires another download to make it available.

In the next chapter, we'll start our exploration of Python with class definition. We'll focus specifically on how objects are initialized using the __init__() special method.