Identifying risks and issues in systems is a fundamental activity when testing any system. In this blog on risk-based testing, we look at how risk translates into actual testing activities and tests can actually depend on many factors such as; the industry you are working in, the type of system you are working on, the stakeholders view of the system, and the time you have to test.
What is risk-based testing?
Risk-based testing prioritises the testing that is performed by the risk of failure and the impact of that failure. In order for this to be possible, an assessment of the risk must be possible based on the context of the project or release of software. Typical considerations when assessing risk include:
- The context of the industry
- The context of the project
- The concerns of the stakeholders / users
- Business risks
- Technical risks
- Regulatory requirements
- Common / previous issues
The degree to which you perform this analysis also depends on the context. If you are working on a system which is mission critical you may spend much more time on this analysis than a minor release to an eCommerce website. In both these examples they would be constrained by time relative to the scale of the project, so risk-based testing would always be required.
To best-manage these risks you would typically record and assess the risks in one place such as in a log, register or matrix. From this you are then able to make an informed decision based on testing scope and priority.
Examples of risk to consider in risk-based testing
Identifying risks is generally the hardest part of the process. You can either start by trying to identify potential risks or start by examining the system and context to identify what could go wrong. More often than not you should look to do both.
You may be alerted to a risk by developers with concerns about the stability of a section of code e.g. relating to input validation which could prevent users from continuing. This would lead you to increasing the priority of testing in this area.
A previous issue in live regarding the performance of a system may instantly increase the perceived risk on another unrelated system. You would take this potential risk and examine the system and context to determine the likelihood and impact of it being affected.
Device, operating system and browser compatibility risks
Seeing patterns in device, operating system and browser compatibility are common, resulting in a number of specific device and browser combinations, for example, consistently exhibiting issues. You should always prioritise devices that have exhibited issues previously, as well as using sound analysis around usage patterns.
Revenue-driving functionality risks
This is an example of inside-out risk analysis. Here you would spend time identifying the key parts of the system that drive revenue, then you would analyse in detail and focus testing efforts around them e.g. usability issues causing a low conversion rate in a loan application system.
Peak load risks
Peak loads can lead to an increase in risk to quality and user-experience e.g. a retail site experiencing dramatically increased traffic on Black Friday. Here there are risks attached to not only how the system may handle such peaks in usage but also in understanding what the anticipated usage will be. You can learn more here about performance load modelling in our recent blog.
For most projects there is a trade-off between the level of quality assurance required and the resource available. A risk-based approach to testing enables you to prioritise and guide testing activities – be it determining the types of testing to scope, or prioritising the order in which the tests will be executed – in order to ascertain the quality of your software and ensure it is fit-for-purpose.
It should be noted that a risk-based test approach should be a continuous process of assessing existing and new risks as new intelligence is received e.g. a given test result, stakeholder feedback, inspection of code, new requirements, or experience of an issue elsewhere.