Taking a harmonized approach to SAP S/4HANA testing tools

My primary premise is that the best way to leverage the tools you need for effective test automation is with a harmonized tools architecture. But what do I mean by this?

Testing change programs

Let’s look first at the impact on testing of moving to SAP S/4HANA. In fact, it could be any large complex program change, not just S/4HANA. More and more of Sogeti’s customers are starting to implement testing around these big change programs and they need a clear view of their testing tools. What are they using? What do they need going forward? How are they (or will they be) integrated to deliver Quality Assurance?

A lack of harmonization in the testing tools prevents this clarity. And without this clarity, you don’t have control. I often tell my customers that trust is good, but control is better. After all, you cannot control what you can’t measure – and if there’s no control, how can you trust the outcome? You need to be able to measure KPIs; to ascertain how and what you’re doing.

A cocktail of tools

The challenge we see emerging, however, is that the use of testing tools across a typical organization’s landscape is not standardized. With a lack of tools harmonization, teams and projects start using their own tools, or use the same tools, but in their own way. They might have inherited certain tools, adding to these with new ones to create a cocktail of testing tools. We have encountered examples where one part of the organization isn’t aware of the test tools used by the other part and buys new tools.

I hope you can begin to see the picture that I’m painting. It’s one of the multiple testing tools and approaches – both automated and manual – that fails to offer the business assurance needed during major change programs, such as the move to SAP S/4HANA.

So, what’s the answer? As I stated at the outset, a harmonized testing tools architecture is what’s required. This is essentially an agreed set of tools and approaches adopted enterprise-wide and, where possible, by the extended supplier ecosystem of ICT vendors and partners. With a single and harmonized toolset, you can look at the bigger picture in terms of quality. At Sogeti, we typically recommend one primary test management master tool sitting on the top of a tool stack. This might be Micro Focus ALM, SAP Solution Manager, or an open-source option like Jira with Zephyr. This is then supported by a suite of other tools, such as ones for test automation, security testing, and non-functional testing.

Everyone should be using these same tools in the same way, making the measurement of agreed KPIs a valid metric of end-to-end quality. I truly believe that shared KPIs (test automation coverage, test effectiveness, testing downtime, etc.) within your harmonized testing tools architecture can drive testing transformation. For example, if you have a KPI around application defects, it becomes possible to use clever predictive algorithms to manage quality around the part of an app that frequently needs fixing. You can plan to build test use cases specifically for a consistently failing piece of software.

Bringing vendors on board

Let me put this into context with an example at one of Sogeti’s clients. The organization was undergoing an SAP transformation program and we identified a disconnect between the testing tools used by three different vendors. One was using Jira and Excel; another was using Solution Manager, and the third contacted people simply by emails. Issues were falling between the cracks, with nobody taking responsibility for fixing them. We deployed a Micro Focus testing tool from the Sogeti OneShare cloud platform as the standard option for tracking and fixing defects – and we insisted that all vendors used it. Within four weeks, we all had clarity on all the problems and the turnaround on fixing defects was reduced significantly with all parties sharing the same information in the cloud.

Of course, one of the challenges in agreeing on a harmonized testing tools architecture is that there are so many options to choose from. Some come with a high price label, while others are free. It’s easy to see why different project teams ‘go their own way’ and how a lack of control over the tools environment can creep in. One approach successfully adopted by several of our clients is to enable enterprise-wide usage of an agreed testing tools to stack on a pay-as-you-go model. This means that the cost of tools used can be correctly allocated on a project-by-project basis by the client’s Project Management Organization

In another project for a client, we ran a proof-of-concept whereby the testing tools architecture, test cases, and test automation scripts were all managed within the single harmonized architecture based on Solution Manager 7.2 and Worksoft Certify for a very large SAP S/4HANA transformation program. We used this as a baseline to build powerful BI reports, analyzing the test data and plugging in the ‘R’ Machine Learning Language. We identified key interfaces that had consistently failed over a period of three years, enabling our client to better plan test cases around the most sensitive interfaces. A simple word cloud from the defect descriptions goes a long way in identifying “the usual suspects”.

The cloud as a testing tools enabler

I touched on the potential of cloud services earlier and I want to finish this blog with another client example featuring Sogeti’s own OneShare cloud. Our client had a fully integrated tools architecture based on Micro Focus, with the ability to add tools on top. We set up this architecture in the cloud and hosted it on OneShare.

The tools architecture harmonized on a single stack of tools gave us full visibility for end-to-end quality. By providing our client with a clear view of the testing set-up across its vendors, we gave the business control over its testing landscape. In one instance, the data we captured on the organization’s application maintenance enabled our client to make a business case for test automation as-a-service that would deliver both cost savings and gains in speed.

Harmonization also meant that all the client’s vendors and other users had to align with the chosen toolset. We provided training on the selected testing tools to ensure their full potential could be realized. As another value-add, by drawing down tools hosted in the Sogeti cloud, our client moved from a CapEx budget model to a wholly OpEx one.

This is the second in a series of blogs I’m writing on testing for the SAP S/4HANA landscape that many of our clients are moving to. I hope it has given you food for thought.

For information on Sogeti’s SAP S/4HANA Business Assurance services, please get in touch.