Tests are necessary, first of all, in order to convince everyone (including ourselves) that the program behaves as it should in test situations. Secondly, they ensure the performance of the code covered by tests in the future. Writing tests is a useful process, because during tests it is very often possible to stumble on problem areas, recall some extreme cases, see problems with interfaces, etc.

By developing of any systems, you need to remember at least three types of tests:

  • Unit tests – tests that verify that functions do what is needed. 
  • Integration tests – tests that verify that several functions together do what is needed. 
  • System tests – tests that verify that the whole system does what is needed.

In one of the posts from google, was published s table with the characteristics of three tests types. “Small”, “Medium” and “Large”.

Unit tests

Unit tests correspond to small tests – they should be fast and only check the correctness  of specific program parts. They should not access the database or work in complex multi-threaded environments. They control compliance with specifications / standards, often they have the role of regression tests.

Integration tests

Integration tests are those tests that can affect several modules and functions. Such tests require more time and sometimes special environments. They are necessary to make sure that individual modules and functions can work with each other. It means, that unit tests verify the conformity of real interfaces to the expected, and integration tests – that functions and modules interact correctly with each other.

System tests

This is the highest level of automatic testing. System tests verify that the whole system works, that its parts perform their tasks and are able to interact correctly.

Why should you keep tracking types

Usually, with the growth of the project, the code base will also grow. The duration of automatic checks will increase, supporting a large number of integration and system tests will become more and more difficult. Therefore, the challenge for developers is to minimize the necessary tests. To do this, try to use unit tests where possible and reduce integration using “mocks”.

Reality

Typical API Test

From the official documentation of flask, we get a ready-made recipe for initializing the application and creating the database. Here is the work with the database. This is not a unit test and not a system test. This is an integration test that uses a database test application.

Why integration and not unit? Because in the processing of requests,is performed interaction with flask, ORM, with our business logic. Handlers act as a unifying element of other parts of the project, so writing of unit tests for them is not too easy (you need to replace the database with mocks, internal logic) and not too practical (will integration tests check similar aspects – “were the necessary functions called?”, ” were the data correctly received? “, etc.).

Names and grouping of tests

In this test, are met all conditions for the “small” tests – is checked the behavior of the function without dependencies for compliance with the expected. But the design raises questions.

It is good practice to write tests that focus on a specific aspect of the program. In this example, there are different functions – validate_password_format, validate_password_format, validate_datetime. Grouping checks is not based on the result, but on the test objects.

The name of the test (test_not_empty_errors) does not describe the test object (which method is being tested), it describes only the result (errors are not empty). This method should be called test__validate_not_empty__error_on_empty. This name describes what is being tested and what result is expected. This should be applied to almost every test name in the project due to the fact to don’t waste the time to discuss the test naming conventions.

Regression tests

def test_datetime_errors (): assert validate_datetime (‘datetime’, ‘0123-24-31T; 431’) == (‘datetime is invalid’,) assert validate_datetime (‘datetime’, ‘2018-10-18T20: 21: 21+ -23: 1 ‘) == (‘ datetime is invalid ‘,) assert validate_datetime (‘ datetime ‘,’ 2015-13-20T20: 20: 20 + 20: 20 ‘) == (‘ datetime is invalid ‘,) assert validate_datetime (‘datetime’, ‘2015-02-29T20: 20: 20 + 20: 20’) == (‘datetime is invalid’,) assert validate_datetime (‘datetime’, ‘2015-12-20T25: 20: 20+ 20:20 ‘) == (‘ datetime is invalid ‘,) assert validate_datetime (‘ datetime ‘,’ 2015-12-20T20: 61: 20 + 22: 20 ‘) == (‘ datetime is invalid ‘,) assert validate_datetime (‘datetime’, ‘2015-12-20T20: 20: 61 + 20: 20’) == (‘datetime is invalid’,) assert validate_datetime (‘datetime’, ‘2015-12-20T20: 20: 20 + 25 : 20 ‘) == (‘ datetime is invalid ‘,) assert validate_datetime (‘ datetime ‘,’ 2015-12-20T20: 20: 20 + 20: 61 ‘) == (‘ datetime is invalid ‘,) assert validate_datetime ( ‘datetime’, ‘2015-13-35T25: 61: 61 + 61: 61’) == (‘datetime is invalid’,)
This test originally consisted of the first two assert. After that, was discovered a “bug” – instead of checking the date, was checked only the regular expression, i.e. 9999-99-99 was considered a normal date. The developer fixed it. Naturally, after fixing the bug, you need to add tests to prevent future regression. Instead of adding a new test in which to write why this test exists, checks have been added to this test.

How should a new test be called, in which we must add verification? Probably test__validate_datetime__error_on_bad_datetime.

Ignoring tools

def test_get_providers (): class Tmp: def __init __ (self, id_external, token, username): self.id_external = id_external self.token = token self.username = username …
Tmp? This is a substitution for an object that is not used in this test. The developer doesn’t seem to know about the existence of @patch and MagicMock from unittest.mock. No need to complicate the code, when there are more adequate tools.

There is such a test that initializes the services (in the database), uses the application context.
def test_get_posts (client): def fake_request (* args, ** kwargs): return [one, two] handler = VKServiceHandler () handler.request = fake_request services_init () with app.app_context (): posts = handler.get_posts (None ) assert len ​​(posts) == 2
You can exclude database and context from the test by simply adding one @patch.
@patch (“mobius.services.service_vk.Service”) def test_get_posts (mock): def fake_request (* args, ** kwargs): return [one, two] handler = VKServiceHandler () handler.request = fake_request posts = handler. get_posts (None) assert len ​​(posts) == 2

Summary

To develop quality software, you need to write tests. At a minimum, to make sure that you write what you need. For large information systems, tests are even more important – they allow you to avoid unwanted interface changes or return bugs. To ensure that written tests do not turn into a lot of strange methods over time, you need to pay attention to the convention tests naming , adhere to good practices, minimize tests. Unit tests can be an excellent tool during development. They can be run after every small change to make sure nothing is broken.

A very important point is that the tests do not guarantee the availability or absence of bugs. Tests ensure that the real result of the program (or part of it) is expected. In this case, verification only occurs for those aspects for which tests were written. Therefore, while creating a quality product, we should not forget about other types of testing.