In this article by Siddharta Govindaraj, author of the book Test-Driven Python Development, we will look at the Event class. The Event class is very simple: receivers can register with the event to be notified when the event occurs. When the event fires, all the receivers are notified of the event.
(For more resources related to this topic, see here.)
A more detailed description is as follows:
Event classes have a connect method, which takes a method or function to be called when the event fires
When the fire method is called, all the registered callbacks are called with the same parameters that are passed to the fire method
Writing tests for the connect method is fairly straightforward—we just need to check that the receivers are being stored properly. But, how do we write the tests for the fire method? This method does not change any state or store any value that we can assert on. The main responsibility of this method is to call other methods. How do we test that this is being done correctly?
This is where mock objects come into the picture. Unlike ordinary unit tests that assert on object state, mock objects are used to test that the interactions between multiple objects occurs as it should.
Hand writing a simple mock
To start with, let us look at the code for the Event class so that we can understand what the tests need to do. The following code is in the file event.py in the source directory:
class Event:
"""A generic class that provides signal/slot functionality"""
def __init__(self):
self.listeners = []
def connect(self, listener):
self.listeners.append(listener)
def fire(self, *args, **kwargs):
for listener in self.listeners:
listener(*args, **kwargs)
The way this code works is fairly simple. Classes that want to get notified of the event should call the connect method and pass a function. This will register the function for the event. Then, when the event is fired using the fire method, all the registered functions will be notified of the event. The following is a walk-through of how this class is used:
>>> def handle_event(num):
... print("I got number {0}".format(num))
...
>>> event = Event()
>>> event.connect(handle_event)
>>> event.fire(3)
I got number 3
>>> event.fire(10)
I got number 10
As you can see, every time the fire method is called, all the functions that registered with the connect method get called with the given parameters.
So, how do we test the fire method? The walk-through above gives a hint. What we need to do is to create a function, register it using the connect method, and then verify that the method got notified when the fire method was called. The following is one way to write such a test:
import unittest
from ..event import Event
class EventTest(unittest.TestCase):
def test_a_listener_is_notified_when_an_event_is_raised(self):
called = False
def listener():
nonlocal called
called = True
event = Event()
event.connect(listener)
event.fire()
self.assertTrue(called)
Put this code into the test_event.py file in the tests folder and run the test. The test should pass. The following is what we are doing:
First, we create a variable named called and set it to False.
Next, we create a dummy function. When the function is called, it sets called to True.
Finally, we connect the dummy function to the event and fire the event.
If the dummy function was successfully called when the event was fired, then the called variable would be changed to True, and we assert that the variable is indeed what we expected.
The dummy function we created above is an example of a mock. A mock is simply an object that is substituted for a real object in the test case. The mock then records some information such as whether it was called, what parameters were passed, and so on, and we can then assert that the mock was called as expected.
Talking about parameters, we should write a test that checks that the parameters are being passed correctly. The following is one such test:
def test_a_listener_is_passed_right_parameters(self):
params = ()
def listener(*args, **kwargs):
nonlocal params
params = (args, kwargs)
event = Event()
event.connect(listener)
event.fire(5, shape="square")
self.assertEquals(((5, ), {"shape":"square"}), params)
This test is the same as the previous one, except that it saves the parameters that are then used in the assert to verify that they were passed properly.
At this point, we can see some repetition coming up in the way we set up the mock function and then save some information about the call. We can extract this code into a separate class as follows:
class Mock:
def __init__(self):
self.called = False
self.params = ()
def __call__(self, *args, **kwargs):
self.called = True
self.params = (args, kwargs)
Once we do this, we can use our Mock class in our tests as follows:
class EventTest(unittest.TestCase):
def test_a_listener_is_notified_when_an_event_is_raised(self):
listener = Mock()
event = Event()
event.connect(listener)
event.fire()
self.assertTrue(listener.called)
def test_a_listener_is_passed_right_parameters(self):
listener = Mock()
event = Event()
event.connect(listener)
event.fire(5, shape="square")
self.assertEquals(((5, ), {"shape": "square"}), listener.params)
What we have just done is to create a simple mocking class that is quite lightweight and good for simple uses. However, there are often times when we need much more advanced functionality, such as mocking a series of calls or checking the order of specific calls. Fortunately, Python has us covered with the unittest.mock module that is supplied as a part of the standard library.
Using the Python mocking framework
The unittest.mock module provided by Python is an extremely powerful mocking framework, yet at the same time it is very easy to use.
Let us redo our tests using this library. First, we need to import the mock module at the top of our file as follows:
from unittest import mock
Next, we rewrite our first test as follows:
class EventTest(unittest.TestCase):
def test_a_listener_is_notified_when_an_event_is_raised(self):
listener = mock.Mock()
event = Event()
event.connect(listener)
event.fire()
self.assertTrue(listener.called)
The only change that we've made is to replace our own custom Mock class with the mock.Mock class provided by Python. That is it. With that single line change, our test is now using the inbuilt mocking class.
The unittest.mock.Mock class is the core of the Python mocking framework. All we need to do is to instantiate the class and pass it in where it is required. The mock will record if it was called in the called instance variable.
How do we check that the right parameters were passed? Let us look at the rewrite of the second test as follows:
def test_a_listener_is_passed_right_parameters(self):
listener = mock.Mock()
event = Event()
event.connect(listener)
event.fire(5, shape="square")
listener.assert_called_with(5, shape="square")
The mock object automatically records the parameters that were passed in. We can assert on the parameters by using the assert_called_with method on the mock object. The method will raise an assertion error if the parameters don't match what was expected. In case we are not interested in testing the parameters (maybe we just want to check that the method was called), then we can pass the value mock.ANY. This value will match any parameter passed.
There is a subtle difference in the way normal assertions are called compared to assertions on mocks. Normal assertions are defined as a part of the unittest.Testcase class. Since our tests inherit from that class, we call the assertions on self, for example, self.assertEquals. On the other hand, the mock assertion methods are a part of the mock object, so you call them on the mock object, for example, listener.assert_called_with.
Mock objects have the following four assertions available out of the box:
assert_called_with: This method asserts that the last call was made with the given parameters
assert_called_once_with: This assertion checks that the method was called exactly once and was with the given parameters
assert_any_call: This checks that the given call was made at some point during the execution
assert_has_calls: This assertion checks that a list of calls occurred
The four assertions are very subtly different, and that shows up when the mock has been called more than one. The assert_called_with method only checks the last call, so if there was more than one call, then the previous calls will not be asserted. The assert_any_call method will check if a call with the given parameters occurred anytime during execution. The assert_called_once_with assertion asserts for a single call, so if the mock was called more than once during execution, then this assert would fail. The assert_has_calls assertion can be used to assert that a set of calls with the given parameters occurred. Note that there might have been more calls than what we checked for in the assertion, but the assertion would still pass as long as the given calls are present.
Let us take a closer look at the assert_has_calls assertion. Here is how we can write the same test using this assertion:
def test_a_listener_is_passed_right_parameters(self):
listener = mock.Mock()
event = Event()
event.connect(listener)
event.fire(5, shape="square")
listener.assert_has_calls([mock.call(5, shape="square")])
The mocking framework internally uses _Call objects to record calls. The mock.call function is a helper to create these objects. We just call it with the expected parameters to create the required call objects. We can then use these objects in the assert_has_calls assertion to assert that the expected call occurred.
This method is useful when the mock was called multiple times and we want to assert only some of the calls.
Mocking objects
While testing the Event class, we only needed to mock out single functions. A more common use of mocking is to mock a class.
Take a look at the implementation of the Alert class in the following:
class Alert:
"""Maps a Rule to an Action, and triggers the action if the rule
matches on any stock update"""
def __init__(self, description, rule, action):
self.description = description
self.rule = rule
self.action = action
def connect(self, exchange):
self.exchange = exchange
dependent_stocks = self.rule.depends_on()
for stock in dependent_stocks:
exchange[stock].updated.connect(self.check_rule)
def check_rule(self, stock):
if self.rule.matches(self.exchange):
self.action.execute(self.description)
Let's break down how this class works as follows:
The Alert class takes a Rule and an Action in the initializer.
When the connect method is called, it takes all the dependent stocks and connects to their updated event.
The updated event is an instance of the Event class that we saw earlier. Each Stock class has an instance of this event, and it is fired whenever a new update is made to that stock.
The listener for this event is the self.check_rule method of the Alert class.
In this method, the alert checks if the new update caused a rule to be matched.
If the rule matched, it calls the execute method on the Action. Otherwise, nothing happens.
This class has a few requirements, as shown in the following, that need to be met. Each of these needs to be made into a unit test.
If a stock is updated, the class should check if the rule matches
If the rule matches, then the corresponding action should be executed
If the rule doesn't match, then nothing happens
There are a number of different ways in which we could test this; let us go through some of the options.
The first option is not to use mocks at all. We could create a rule, hook it up to a test action, and then update the stock and verify that the action was executed. The following is what such a test would look like:
import unittest
from datetime import datetime
from unittest import mock
from ..alert import Alert
from ..rule import PriceRule
from ..stock import Stock
class TestAction:
executed = False
def execute(self, description):
self.executed = True
class AlertTest(unittest.TestCase):
def test_action_is_executed_when_rule_matches(self):
exchange = {"GOOG": Stock("GOOG")}
rule = PriceRule("GOOG", lambda stock: stock.price > 10)
action = TestAction()
alert = Alert("sample alert", rule, action)
alert.connect(exchange)
exchange["GOOG"].update(datetime(2014, 2, 10), 11)
self.assertTrue(action.executed)
This is the most straightforward option, but it requires a bit of code to set up and there is the TestAction that we need to create just for the test case.
Instead of creating a test action, we could instead replace it with a mock action. We can then simply assert on the mock that it got executed. The following code shows this variation of the test case:
def test_action_is_executed_when_rule_matches(self):
exchange = {"GOOG": Stock("GOOG")}
rule = PriceRule("GOOG", lambda stock: stock.price > 10)
action = mock.MagicMock()
alert = Alert("sample alert", rule, action)
alert.connect(exchange)
exchange["GOOG"].update(datetime(2014, 2, 10), 11)
action.execute.assert_called_with("sample alert")
A couple of observations about this test:
If you notice, alert is not the usual Mock object that we have been using so far, but a MagicMock object. A MagicMock object is like a Mock object but it has special support for Python's magic methods which are present on all classes, such as __str__, hasattr. If we don't use MagicMock, we may sometimes get errors or strange behavior if the code uses any of these methods. The following example illustrates the difference:
>>> from unittest import mock
>>> mock_1 = mock.Mock()
>>> mock_2 = mock.MagicMock()
>>> len(mock_1)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: object of type 'Mock' has no len()
>>> len(mock_2)
0
>>>
In general, we will be using MagicMock in most places where we need to mock a class. Using Mock is a good option when we need to mock stand alone functions, or in rare situations where we specifically don't want a default implementation for the magic methods.
The other observation about the test is the way methods are handled. In the test above, we created a mock action object, but we didn't specify anywhere that this mock class should contain an execute method and how it should behave. In fact, we don't need to. When a method or attribute is accessed on a mock object, Python conveniently creates a mock method and adds it to the mock class. Therefore, when the Alert class calls the execute method on our mock action object, that method is added to our mock action. We can then check that the method was called by asserting on action.execute.called.
The downside of Python's behavior of automatically creating mock methods when they are accessed is that a typo or change in interface can go unnoticed.
For example, suppose we rename the execute method in all the Action classes to run. But if we run our test cases, it still passes. Why does it pass? Because the Alert class calls the execute method, and the test only checks that the execute method was called, which it was. The test does not know that the name of the method has been changed in all the real Action implementations and that the Alert class will not work when integrated with the actual actions.
To avoid this problem, Python supports using another class or object as a specification. When a specification is given, the mock object only creates the methods that are present in the specification. All other method or attribute accesses will raise an error.
Specifications are passed to the mock at initialization time via the spec parameter. Both the Mock as well as MagicMock classes support setting a specification. The following code example shows the difference when a spec parameter is set compared to a default Mock object:
>>> from unittest import mock
>>> class PrintAction:
... def run(self, description):
... print("{0} was executed".format(description))
...
>>> mock_1 = mock.Mock()
>>> mock_1.execute("sample alert") # Does not give an error
<Mock name='mock.execute()' id='54481752'>
>>> mock_2 = mock.Mock(spec=PrintAction)
>>> mock_2.execute("sample alert") # Gives an error
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:Python34libunittestmock.py", line 557, in __getattr__
raise AttributeError("Mock object has no attribute %r" % name)
AttributeError: Mock object has no attribute 'execute'
Notice in the above example that mock_1 goes ahead and executes the execute method without any error, even though the method has been renamed in the PrintAction. On the other hand, by giving a spec, the method call to the nonexistent execute method raises an exception.
Mocking return values
The second variant above showed how we could use a mock Action class in the test instead of a real one. In the same way, we can also use a mock rule instead of creating a PriceRule in the test. The alert calls the rule to see whether the new stock update caused the rule to be matched. What the alert does depends on whether the rule returned True or False.
All the mocks we've created so far have not had to return a value. We were just interested in whether the right call was made or not. If we mock the rule, then we will have to configure it to return the right value for the test. Fortunately, Python makes that very simple to do.
All we have to do is to set the return value as a parameter in the constructor to the mock object as follows:
>>> matches = mock.Mock(return_value=True)
>>> matches()
True
>>> matches(4)
True
>>> matches(4, "abcd")
True
As we can see above, the mock just blindly returns the set value, irrespective of the parameters. Even the type or number of parameters is not considered. We can use the same procedure to set the return value of a method in a mock object as follows:
>>> rule = mock.MagicMock()
>>> rule.matches = mock.Mock(return_value=True)
>>> rule.matches()
True
>>>
There is another way to set the return value, which is very convenient when dealing with methods in mock objects. Each mock object has a return_value attribute. We simply set this attribute to the return value and every call to the mock will return that value, as shown in the following:
>>> from unittest import mock
>>> rule = mock.MagicMock()
>>> rule.matches.return_value = True
>>> rule.matches()
True
>>>
In the example above, the moment we access rule.matches, Python automatically creates a mock matches object and puts it in the rule object. This allows us to directly set the return value in one statement without having to create a mock for the matches method.
Now that we've seen how to set the return value, we can go ahead and change our test to use a mocked rule object, as shown in the following:
def test_action_is_executed_when_rule_matches(self):
exchange = {"GOOG": Stock("GOOG")}
rule = mock.MagicMock(spec=PriceRule)
rule.matches.return_value = True
rule.depends_on.return_value = {"GOOG"}
action = mock.MagicMock()
alert = Alert("sample alert", rule, action)
alert.connect(exchange)
exchange["GOOG"].update(datetime(2014, 2, 10), 11)
action.execute.assert_called_with("sample alert")
There are two calls that the Alert makes to the rule: one to the depends_on method and the other to the matches method. We set the return value for both of them and the test passes.
In case no return value is explicitly set for a call, the default return value is to return a new mock object. The mock object is different for each method that is called, but is consistent for a particular method. This means if the same method is called multiple times, the same mock object will be returned each time.
Mocking side effects
Finally, we come to the Stock class. This is the final dependency of the Alert class. We're currently creating Stock objects in our test, but we could replace it with a mock object just like we did for the Action and PriceRule classes.
The Stock class is again slightly different in behavior from the other two mock objects. The update method doesn't just return a value—it's primary behavior in this test is to trigger the updated event. Only if this event is triggered will the rule check occur.
In order to do this, we must tell our mock stock class to fire the event when the update event is called. Mock objects have a side_effect attribute to enable us to do just this.
There are many reasons we might want to set a side effect. Some of them are as follows:
We may want to call another method, like in the case of the Stock class, which needs to fire the event when the update method is called.
To raise an exception: this is particularly useful when testing error situations. Some errors such as a network timeout might be very difficult to simulate, and it is better to test using a mock that simply raises the appropriate exception.
To return multiple values: these may be different values each time the mock is called, or specific values, depending on the parameters passed.
Setting the side effect is just like setting the return value. The only difference is that the side effect is a lambda function. When the mock is executed, the parameters are passed to the lambda function and the lambda is executed. The following is how we would use this with a mocked out Stock class:
def test_action_is_executed_when_rule_matches(self):
goog = mock.MagicMock(spec=Stock)
goog.updated = Event()
goog.update.side_effect = lambda date, value:
goog.updated.fire(self)
exchange = {"GOOG": goog}
rule = mock.MagicMock(spec=PriceRule)
rule.matches.return_value = True
rule.depends_on.return_value = {"GOOG"}
action = mock.MagicMock()
alert = Alert("sample alert", rule, action)
alert.connect(exchange)
exchange["GOOG"].update(datetime(2014, 2, 10), 11)
action.execute.assert_called_with("sample alert")
So what is going on in that test?
First, we create a mock of the Stock class instead of using the real one.
Next, we add in the updated event. We need to do this because the Stock class creates the attribute at runtime in the __init__ scope. Because the attribute is set dynamically, MagicMock does not pick up the attribute from the spec parameter. We are setting an actual Event object here. We could set it as a mock as well, but it is probably overkill to do that.
Finally, we set the side effect for the update method in the mock stock object. The lambda takes the two parameters that the method does. In this particular example, we just want to fire the event, so the parameters aren't used in the lambda. In other cases, we might want to perform different actions based on the values of the parameters. Setting the side_effect attribute allows us to do that.
Just like with the return_value attribute, the side_effect attribute can also be set in the constructor.
Run the test and it should pass.
The side_effect attribute can also be set to an exception or a list. If it is set to an exception, then the given exception will be raised when the mock is called, as shown in the following:
>>> m = mock.Mock()
>>> m.side_effect = Exception()
>>> m()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:Python34libunittestmock.py", line 885, in __call__
return _mock_self._mock_call(*args, **kwargs)
File "C:Python34libunittestmock.py", line 941, in _mock_call
raise effect
Exception
If it is set to a list, then the mock will return the next element of the list each time it is called. This is a good way to mock a function that has to return different values each time it is called, as shown in the following:
>>> m = mock.Mock()
>>> m.side_effect = [1, 2, 3]
>>> m()
1
>>> m()
2
>>> m()
3
>>> m()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:Python34libunittestmock.py", line 885, in __call__
return _mock_self._mock_call(*args, **kwargs)
File "C:Python34libunittestmock.py", line 944, in _mock_call
result = next(effect)
StopIteration
As we have seen, the mocking framework's method of handling side effects using the side_effect attribute is very simple, yet quite powerful.
How much mocking is too much?
In the previous few sections, we've seen the same test written with different levels of mocking. We started off with a test that didn't use any mocks at all, and subsequently mocked out each of the dependencies one by one. Which one of these solutions is the best?
As with many things, this is a point of personal preference. A purist would probably choose to mock out all dependencies. My personal preference is to use real objects when they are small and self-contained. I would not have mocked out the Stock class. This is because mocks generally require some configuration with return values or side effects, and this configuration can clutter the test and make it less readable. For small, self-contained classes, it is simpler to just use the real object.
At the other end of the spectrum, classes that might interact with external systems, or that take a lot of memory, or are slow are good candidates for mocking out. Additionally, objects that require a lot of dependencies on other object to initialize are candidates for mocking. With mocks, you just create an object, pass it in, and assert on parts that you are interested in checking. You don't have to create an entirely valid object.
Even here there are alternatives to mocking. For example, when dealing with a database, it is common to mock out the database calls and hardcode a return value into the mock. This is because the database might be on another server, and accessing it makes the tests slow and unreliable. However, instead of mocks, another option could be to use a fast in-memory database for the tests. This allows us to use a live database instead of a mocked out database. Which approach is better depends on the situation.
Mocks versus stubs versus fakes versus spies
We've been talking about mocks so far, but we've been a little loose on the terminology. Technically, everything we've talked about falls under the category of a test double. A test double is some sort of fake object that we use to stand in for a real object in a test case.
Mocks are a specific kind of test double that record information about calls that have been made to it, so that we can assert on them later.
Stubs are just an empty do-nothing kind of object or method. They are used when we don't care about some functionality in the test. For example, imagine we have a method that performs a calculation and then sends an e-mail. If we are testing the calculation logic, we might just replace the e-mail sending method with an empty do-nothing method in the test case so that no e-mails are sent out while the test is running.
Fakes are a replacement of one object or system with a simpler one that facilitates easier testing. Using an in-memory database instead of the real one, or the way we created a dummy TestAction earlier in this article would be examples of fakes.
Finally, spies are objects that are like middlemen. Like mocks, they record the calls so that we can assert on them later, but after recording, they continue execution to the original code. Spies are different from the other three in the sense that they do not replace any functionality. After recording the call, the real code is still executed. Spies sit in the middle and do not cause any change in execution pattern.
Summary
In this article, you looked at how to use mocks to test interactions between objects. You saw how to hand write our own mocks, followed by using the mocking framework provided in the Python standard library.
Resources for Article:
Further resources on this subject:
Analyzing a Complex Dataset [article]
Solving problems – closest good restaurant [article]
Importing Dynamic Data [article]
Read more
United States
Great Britain
India
Germany
France
Canada
Russia
Spain
Brazil
Australia
South Africa
Thailand
Ukraine
Switzerland
Slovakia
Luxembourg
Hungary
Romania
Denmark
Ireland
Estonia
Belgium
Italy
Finland
Cyprus
Lithuania
Latvia
Malta
Netherlands
Portugal
Slovenia
Sweden
Argentina
Colombia
Ecuador
Indonesia
Mexico
New Zealand
Norway
South Korea
Taiwan
Turkey
Czechia
Austria
Greece
Isle of Man
Bulgaria
Japan
Philippines
Poland
Singapore
Egypt
Chile
Malaysia
