In this article I will discuss the software tests that can be\u00a0carried out\u00a0at the system level. These tests are an essential link in the continuous integration chain, they may or may not be functional and have very specific characteristics. I will discuss\u00a0best\u00a0practices to be followed and implementation issues, based on the example of system tests of an API.\u00a0So,\u00a0let\u2019s see what the test levels are and what we call a system test.<\/p>\n
The test levels<\/h2>\n
In accordance with the recommendations of the ISTQB (International Software Testing Qualification Board<\/a>), software tests are classified into different levels. Here are the traditional test levels, which are found in the classic test pyramid:<\/p>\n At AT Internet, as our software solution is very extensive and complex, we have chosen to add two levels of testing: <\/p>\n These \u201csystem\u201d tests are the first level of the so-called \u201cblack box\u201d tests, i.e. they are defined on the basis of the product specifications (business rules, performance requirements, etc.) without any access to the source code. The so-called \u201cwhite box\u201d tests, on the other hand, are defined on the basis of the internal structure of the product (architecture, code, code portions, etc.).<\/p>\n These tests System In Agility, we have no other option than to automate these tests<\/a>, otherwise continuous integration<\/a> is compromised by longer delivery times due to manual testing phases that can be endless. I will therefore only deal here with automated system tests.<\/p>\n While unit Each System tests are the first tests directly related to the customer\u2019s acceptance criteria in the test process. These tests, if automated, will make it possible to detect possible regressions as early as possible, otherwise there is a risk of unpleasant surprises in later phases of the project: when the system is integrated into the complete solution or worse, at the customer\u2019s or one of their representatives\u2019 premises (product owner for example).<\/p>\n It is then It is not always easy to find the resources to set up and automate this type of test: it requires multiple skills, both in test design and automation development and we are not always lucky enough to have a test developer on the team. In some cases, product developers will be called upon to carry out some of these implementations. <\/p>\n Testing is surely part of the development activities and the responsibility for delivering a product that works is that of the entire team, isn\u2019t it? <\/p>\n To be able to properly test a system, you must first understand various aspects of our system: <\/p>\n These are We often We now know These more or less formally described behaviours will be used directly to describe the different test cases that will take on the role of system tests.<\/p>\n Some system Let\u2019s take In both System Depending In other It can be Non-functional aspects such as performance or resilience can also be tested in system tests, with different techniques and tools depending on the needs.<\/p>\n Let\u2019s take These To design Performance![\"Pyramid](\"https:\/\/blog.atinternet.com\/wp-content\/uploads\/2019\/12\/pyramid-traditional-test-systeme-levels-1.jpg\")
<\/figure>\n<\/div>\n\n
![\"Pyramid](\"https:\/\/blog.atinternet.com\/wp-content\/uploads\/2019\/12\/pyramid-at-internet-test-system-levels.jpg\")
<\/figure>\n<\/div>\n\n
The system tests<\/h2>\n
![\"Black](\"https:\/\/blog.atinternet.com\/wp-content\/uploads\/2019\/12\/blackbox-whitebox-test.jpg\")
<\/figure>\n<\/div>\n
\nmust be completely independent of the internal structure of the system
\n(language and technologies used, implementation details etc\u2026). They must
\nsimply focus on the inputs and outputs of the system, observing it completely
\nobjectively. This is what makes them powerful and useful: they then make it
\npossible to validate a profession or behaviours that are directly perceptible
\nto the customer.<\/p>\n
\ntests are essential when modifying the internal structure of the system
\n(refactoring, replacing internal components, adding functionalities, etc.):
\nthey ensure that these modifications do not lead to regressions for customers.<\/p>\n
\nand integration tests are essential and must be provided as much as possible,
\nit is essential to invest in tests at the system level to ensure that the
\ndesired business is delivered to the customer.<\/p>\n
\nbusiness brick can indeed correctly do what it is designed for, but their
\nassembly can sometimes lead to peculiar behaviour! On the other hand, the job
\ncan be respected while parts of the system are sometimes in poor condition,
\nwhich is still rarer. <\/p>\n
\nmuch more expensive to correct the problems detected and the risk of delaying
\nthe delivery of the project is much greater.<\/p>\nWhat\u2019s the approach to
\ntesting a system?<\/h2>\n![\"Diagram](\"https:\/\/blog.atinternet.com\/wp-content\/uploads\/2019\/12\/system-schema-in-out.jpg\")
<\/figure>\n<\/div>\nThe system entries<\/h3>\n
\nall triggers for system behaviour. We often think of user actions as obvious
\ntriggers, but there may be many others. Here are some examples:<\/p>\n\n
The system outputs<\/h3>\n
\nthink here of the answer provided to the user but there are other very common
\noutputs that we do not always think about. For example:<\/p>\n\n
\nanswer to the user<\/li>\n
\nwriting (logs)<\/li>\n
\ndata to a database<\/li>\n
\nissuance of a notification<\/li>\n<\/ul>\nExpected behaviours (or
\nbusiness rules)<\/h3>\n
\nhow to activate the system by playing on its different inputs. We also know
\nwhat we can observe by looking at the different outings. We then need to know
\nthe relationships between inputs and outputs. These relationships are described
\nas system business rules and can take different forms: from the acceptance
\ncriterion of a user story in Agility to more detailed specifications in other
\ncontexts.<\/p>\nExecution contexts <\/h3>\n
\nbehaviours depend not only on the inputs received but also on the context in
\nwhich the system is located at the time of the test.<\/p>\n
\nthe example of a call to an API to record new information in the system. We
\nthen have two cases, with different imaginable behaviours:<\/p>\n\n
\n
\n
\n
\n
\n
\nsituations, the execution context has a direct influence on the expected
\nbehaviour of the system.\u00a0 This is where
\nthe test dataset takes on its importance: we will generate the context in which
\nwe want to be before each test to be able to validate the different behaviours
\nof our system. We then have a number of combinations of all these elements that
\nwill constitute our test cases:<\/p>\n\n
\ntest data set<\/li>\n
\nor more inputs to \u201cactivate\u201d<\/li>\n
\nbehaviour<\/li>\n
\nor more outputs to be controlled<\/li>\n<\/ul>\n
\ntests can then be implemented. This will require the implementation of a test
\nmechanism that must be able to play on the system inputs and control the
\nvalidity of its behaviour by observing its different outputs:<\/p>\n![\"Test](\"https:\/\/blog.atinternet.com\/wp-content\/uploads\/2019\/12\/test-mecanic-system-monitoring-in-out.jpg\")
<\/figure>\n<\/div>\n
\non the case, this test mechanism is directly made available in tools on the
\nmarket, depending on the system we want to test:<\/p>\n\n
\ncases, it will be necessary to implement a mechanism adapted to your needs,
\nwhich must allow you to play with the different inputs and outputs of the system
\nunder test: <\/p>\n\n
\nin a Kafka topic<\/li>\n
\n\/ retrieving notifications<\/li>\n
\nof data into databases<\/li>\n
\nemails<\/li>\n
\nseen here that the test tool is not linked to the technology used in the system
\nto be tested. It is often very different and is not related to the information
\nprocessing needs of the system but rather to the constraints imposed on the
\nclient(s) of the system.<\/p>\nTesting an API<\/h2>\n
\nthe concrete example of an API as a system to be tested. The main classics of
\nPLC validation are, in the order in which the system must check them:<\/p>\n\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\noperations are increasingly costly for the system, which must try to return an
\nerror as soon as possible<\/u> if it is relevant, without initiating the next
\nsteps, otherwise generating unnecessary load on the servers and thereby
\naltering non-functional aspects of the system, or even its security.<\/p>\n
\nthese different tests, different design techniques can be used, including limit
\nvalue analysis, equivalence classes, state transitions, decision tables, etc. <\/p>\n
\nor safety tests can be considered with different tools such as LOAD UI,
\nGatling, JMeter, Neoload, for example. In this type of test, we find well-known
\ntechniques such as:<\/p>\n