Better Test Strategy

Posted by at 13:20h

Strategy is about using scarce resources wisely. Test capacity is certainly a valuable resource that should be applied thoughtfully. One way of optimizing is to consider two dimensions of the IT portfolio.

  •  Impact of failure
  • Likelihood of failure


The first is focused on business while the second is the technical dimension. Both need to be considered and guide us to allocate most of our testing efforts to the parts of the system which score high in both aspects and least where both score low. This require good IT/Business collaboration which frequently a challenge but not the topic of this blogpost.

Impact assessment is in itself multi-dimensional and varies from organization to organization depending on the industry and overall business environment. For businesses it is most important to understand how failure impacts revenue and costs. The assessment should include future impact, as loss of reputation and potential legal actions resulting from a failure, can have large consequences.

This brings us to the likelihood of failure which has both obvious and not so obvious aspects. The most obvious is always “what have we changed?” This is typically the driver for the test program in most organizations. New applications, new releases, patches, etc. are undergoing tests as change happens and some of the change may not be good.

A less obvious but very important aspect is how the changed component impacts other parts of the IT system. The challenge is to reliably determine which parts of the system should be regression tested outside the recently changed application. Some of these system parts may be easily identified but many are not. The consequences of failing to identify impacted system parts can be huge but the cost of testing is also high so finding the right balance is important but frequently tricky.

Understanding the complex maze of relationships among applications is difficult. The problem get further complicated when you seek to understand the effects of changing some upstream on the further downstream applications. For example: In a bank if you changed the branch number field from 3 digits to 4 digits. This change is huge even though it seems like a simple thing to do. Almost every system will need to be tested for this change. But it’s important to understand what to test and how this field impacts other applications.

Comprehensive understanding of enterprise data is the key to truly understand data relationships. These insights will allow you to identify applications and middleware that is impacted by the recent changes as they consume data that is changed by the application you recently changed or introduced.

Unfortunately, the data landscape is typically not well known in most companies. There are several reasons for this like the incremental development of the total system, mergers, etc. but at the end of the day most companies will need to discover their data in order to make the right judgement about the test strategy.

Discovery of the enterprise data landscape requires good tools. ROKITT is offering an enterprise data manager with the capability to:

  • Automatic discovery of data with relationships intuitively displayed as a graph
  • Natural language query capability provides ease of use and enables effective management
  • Synthetic data generation & sub-setting to improve test coverage and timely availability of test data
  • Data Masking with production quality data to enable high quality test while protecting privacy and property as well as regulatory compliance
  • Database independent solution supporting industry leading databases
  • One-stop shop to create, manage, secure, and refresh data


With tools like ROKITT’s enterprise data manager, organizations will be able to quickly regain understanding of their data landscape and use their new knowledge to improve their test strategy. The benefits of understanding the data are not just limited to test strategy. Architects, designers & developers can visually see the impact of their data in multiple places. The impact will be higher quality at a lower cost.


Image courtesy of Salvatore Vuono at