Testing a set of RESTful Anghami APIs with pytest
Testing is integral in order to provide a good product/service to our users. A broad term in itself, testing’s many facets range from unit to system (or application) level, from at-deployment to 24/7-monitoring of your APIs health (status, response time, response data and its correctness). One particular facet we are interested in, is functional testing. @AnghamiTech, we rely on many ways to test the functionality of our public APIs such as PHPunit, in-house written tools, SaaS like Runscope.com…
While working on a new feature, we found ourselves in need of implementing a complex set of functional tests, that mocked one or more users accessing the same resource server-side, within a set of 20+ consecutive API calls. With SaaS services being limited, access and nbr_of_api_calls/test wise, and having used python for some in-house tools and scripts, we found that pytest offered a good route to get the job done. This article then, explores our 1st flight with pytest. (picture credit to original artist).
Pre-flight Routine Checks & Takeoff: Boilerplate
Pre-flight checks are necessary routine tasks done at the beginning of every flight. While repeating them religiously is required to ensure a flight’s safety and smooth operation, getting rid of such repetitions, religiously if we might say, is our aim here.
Our objective is boilerplate code. We are treating this term loosely, for we are not referring to the verbosity of a certain language, but to code segments that are necessary for our suite and can be modularized. After all, even though the purpose of such a suite is to test our code (public APIs), it itself is code, hence good programming practices also apply to it. This allows it be readable, manageable, easily updatable and most importantly, maintainable. Maintainability must not be an afterthought, but an integral part of your pre-flight plans as one of the most common of problems with test suites is the steep increase in the difficulty of its maintainability vs the complexity and number of the scenarios it contains.
So, in preparation for our 1st flight, helper modules, including wrappers around certain methods from the requests module, were recruited to help us co-pilot:
- Http session management, including adapters, setup, teardown…
- Http Request building and preparation.
- Request sending, exception handling (connection, timeouts…) and retries.
- DataBase connections, transactions and rollbacks.
- Anghami Public API request building, in order to make scenario writing easier and less verbose, especially since a scenario can, and most probably will, include multiple calls to the same API.
In addition to pytest’s conftest.py file, centralizing certain configurations in your own – let’s say – config.py file provides a smoother takeoff, such as:
- DB credentials as dictionaries (host, post, username, schema…).
- User account credentials and device identifiers (dictionaries) for easily simulating multiple users.
- A list of subdomains such as live.myapp.com or staging.myapp.com to easily switch and direct your test traffic to different environments.
Thus, if we are to look at an example takeoff and a short flight, our test_module.py, kept compact and easy-to-follow using the above, would fly like:
Flight Crew & Mid-flight Meals: Fixtures
“Are you hungry Sir? Would like a drink Madame?” May I suggest a plate of dependency injection?
A prime example of dependency injection, “…fixture functions take the role of the injector and test functions are the consumers of fixture objects.” One important aspect of the test we wanted to apply was statefulness, the other was sharing test data between test steps (methods). For our 1st flight, and defined as Classes in their own separate modules, and we opted for 3 custom fixtures:
- A fixture to manage a mocked user’s http session and access token.
- A fixture to manage (create, modify, destroy) the state of the resource (object) we are interested in, in addition to using the local state to assert on the state of this resource once it is posted(created) / modified / destroyed(deleted) server-side.
- A fixture that acts as a temporary holder of data received from the server and passed to the next test step. (more on this particular fixture in the following section of this article)
For our 1st flight, we experimented with the fixtures’ scope and settled on “class”, as grouping the test steps (methods) within a class, pytest-style, made sense. Though for future flights, it would make better sense leveling up the fixture that manages a mocked user’s http session and access token to “session” scope using pytest’s conftest.py file in addition to taking advantage of "yield" statements or finalizers for better cleanup and teardown.
Turbulence & Maneuvers: Parametrization
“Please return to your seats. We will be experiencing turbulence.” We have to maneuver 3 bumps.
First, given that we were asserting on a resource (an object) with a long list of attributes, when one assertion failed, the rest did not get executed, which wasted time and made the error report incomplete. While you might find many modules and 3rd party plugins to maneuver around this bump, we opted for one provided natively: parametrization and a function containing a single assert statement, which allowed us to iterate over all the desired attributes.
However, the above put us on a flight path towards a second bump: iterating over a test method that contained a public API call. It did not make sense to trigger a request for every attribute we wanted to check. So we took advantage of a fixture, (the one mentioned in point 3 in the previous section), stored in it the received response and shared it with the parametrized test method.
Similar to the 2nd bump, the 3rd was a byproduct of the previous one’s solution. For we had to re-run the set of parametrized assertions after every modification done on the object in question. This added, for a lack of a better word and in the context of the 1st section of this article, boilerplate and repetition, not to mention, it turned maintaining the test into a nightmare when the developers modified the object’s structure. Thus we moved the parametrized array of tuples to a separate file, turned it into a template-like resource and made use of the "eval()" function in order to evaluate statements that referred to fixtures or objects that are undeclared within the scope of the file storing them.
Thus if we are to look at an example, maneuvering turbulent weather looked like:
In parametrized.py we have:
And in test_module.py we then have:
Landing & Future Flights
We reached the end of our 1st flight and landed successfully. We found a way to implement some complex functional tests and simulate multiple users that interacted with the same resource server-side the way we needed.
As for future flights, we are charting routes to check out finalizers, output processing and report generation, integration with our work environment, improving the uses of fixture and parametrization for object templating, and some form of test scheduling.
