Fredrik Scheja

"Transforming your Software Testing to fit the modern organisations of the next decade"

Sogeti expert Fredrik Scheja sharing his expertise on the topic and how developers and testers should prepare for this transformative journey.

Michelangelo said, at the age of 85: “I am still learning.”

It is often said that we learn from our mistakes – we learn from all actions in fact. Our ability to learn however rest on our ability to reflect on the outcome from said actions. I'll never stop learning, even this article acts as stepping-stone for my own personal development, and I write primarily for my own sake in order to test my mental models on you, the reader, and possibly see what outcome and effect it might have on you. So please get back to me if you have any sorts of feelings, thoughts or reflections on what I have to say. 

I would like to tell where I strongly believe software testing is heading. This, in order to shine some light into the various trends I see in the industry both from a historical aspect as well as in the future. But please look at this as a brief reflection on the way, the journey is infinite and has no end. I want to explore this subject together with you because it is more fun to do things together and I know I learn more through others. This article might share some light of where my thoughts are right now, and that could perhaps act as a mental stepping-stone for you. After all, I wouldn't be surprised if you might hold the next puzzle-piece I later search for in order to explore this journey further.

We come from an industrial age, where transactional test has been our way-of-working historically. What do I mean by transactional test? It means that we have been focusing a lot of the actions we do, the documents we create, the checklists we follow and the interactions we establish. Our entire software testing world has been orbiting around the process, the routines, the tools we use as if they should solve our problems and issues. But guess what, we are humans and are placed here on earth for a higher purpose. And this is exactly why we now need to indulge into the transformational era - for real. We need to transform our way to plan, talk, educate, design and how we perform and how we recruit within software testing.

I know – the transformational phase might seem dangerous at a first glance, but if you embrace it you´ll develop yourself and your abilities to think, act, see and move faster but at the same time feel stronger, more confident in yourselves and rest assure that you´ll offer more value with less energy. The journey might feel scary sometimes and you might feel lost, because it mandates for you to let go of your current mental models and the way you see yourself - but trust me, it is necessary. The way we have been operating will not scale, it is not sustainable to work the way we've been working and it is only a matter of time until the system collapses. The fourth industrial revolution is building up momentum – fast.

I use models in my line of work and I love models – it comes with the part of the role as an architect. Here is one I've inherited from ‘The Digital Happiness’ report series from SogetiLabs. I’ve added an additional layer to it to explain it from a development process-perspective. Most organizations has left the warm and cozy factory-based transactional development processes where resources were seen as cogs in a complex machine. The problem with these organizational setups were that they performed very slowly. The software development organization is seen as a factory that mass-produces software on a large scale. The setup is best utilized when the requirements for software is static over time and identical from all aspects of end-user perspectives, which is rarely the case in reality. Agile development methodologies of various flavors has therefore become popular and are all grouped together in this model as transformational development approaches because what they all have in common is an aspiration to transform a transactional development model into one future state – the authentic organization. 

No alt text provided for this image

What is an authentic organization? According to the elevator pitch made by Celia V. Harquail: 'An organization is authentic when its actions, its character, and its sense of purpose are aligned with and support each other. An organization can be authentic only when:

  • The organization knows ‘who it is” and what characteristics make it distinctive. People in the organization understand why these distinctive features are meaningful.
  • The organization knows what it is striving to create in this world. It has a sense of purpose that goes beyond the products and services it offers.
  • The organization knows how its distinctiveness fuels and adds unique insight to the pursuit of organization’s purpose.
  • The organization has a few signature actions, behaviors and products that bring to life the organization’s meaning, and move it towards its purpose.
  • Both the purpose and the distinctive, meaningful characteristics of the organization resonate with stakeholders and draw members, customers, and constituencies to the organization.
  • The organization actively supports its members, customers, and constituencies in their own authenticity as they work with the organization to achieve its purpose.'

This state could also be explained through the words of our agile syntax – when the idea of autonomous team has distilled down and transformed into the autonomous human. When each individual in the organisation have a deep understanding of the business goals and how their personal mission and actions relate to these goals. The individuals are also highly self-confident and aware of their own individual ethics and values – which then of course matches the organisation you work from within. This has the capacity to create an enormous flow, creativity and innovation due to the fact that we all are aiming for a joint vision.

But what do we need to do in order to get there as entire organizations? What do we need to focus on, what do we need to learn as individuals, and as a group and business? In order to enable this change at scale we are today focusing a lot on technical aspects like operational AI, automation, big data and more people-focused trends like data-driven decision-making, experimentation, servant leadership and so forth. But what will happen after robots really actually help us doing the transactional tasks?

In the industry today, it is said that the need for people with high Emotional Intelligence (EI) will most likely develop into a 'must-have'-skill for most businesses within the upcoming five years. This means that the market expects intelligent automation and robotics in our organizations to enable humans to focus more on our self- & social awareness and our self- & relationship management. But does this mean that we all need to become socially skilled experts? No, I don´t think so, but I think that the balance will shift gradually so that people with Emotional Intelligence will become more frequent in our teams and they will ask the necessary transformational-focused questions more often – enabling us all to become more self-aware. And – let's face it, we as humans won't cope being hyper-effective each and every minute around the clock. Sometimes it feels good to do robot-work from time to time, but it should be because we want and need it, not perhaps because we´re forced to. 

No alt text provided for this image

But how do this relate to software testing? Firstly, it'll be necessary to change how we look on software testing; from the transactional view as a process that should filter out the bugs into seeing it as an enabler of new insights and learnings. This is a paradigm shift for software testing in its foundation because we need to enable these autonomous humans in this highly effective organisation to make smart, hyper-fast decisions based on intelligence and authentic information rather than guesses and faith. Because, as an effect of these fast-moving organisations – also according to Conway's law – the system landscape will be continuously re-shape and transform in a hyper-fast way. In order to keep this up we need to look upon how we enable these kind of organisations with information on all kinds of software-related quality-focused aspects.

No alt text provided for this image

What would people need to know? What kind of information would they appreciate? Would it be possible to know before-hand? In some cases yes; in many cases no. We need a new narrative of testing in order to better understand how to approach and tackle this new paradigm because our current mental models are not even closely enough.

The five hops of Software Testing

In the book Testing in the Digital Age, by Rik Marselis et al. Five 'hops' of software testing is introduced:

  1. Automation and Robots
  2. Use the Data
  3. Go Model Based
  4. Use Artificial Intelligence
  5. Test Forecasting
No alt text provided for this image

I believe in this model, but I don´t believe these hop will be taken sequentially, I´m certain they will have their own hype that is slightly shifted in time, and even the order of these different 'hype-hops' is going to be interesting to see when they come. I recon these hops will be accompanied by a great deal of cool buzz-words, looking forward to see what they might be. I believe we are currently in the first hop. The market is nowadays screaming for automation experts and implementing robotics everywhere.

Buzz-words are clearly Robotics Process Automation, Articifial Intelligence and Machine Learning. Soon we will enter the second hop where the beauty of development/QA-related data will be widely understood. This is when test automation is going to be seen as a data provider instead of a bug filter. But the data will make us painfully aware of the fact that the complexity of our system landscape far exceeds our wildest imagination. So we enter hop 3 – Go Model Based. Here I foresee that we understand that we'll have to start look upon models in a higher abstraction perspective.

An architecture model consisting of boxes and lines between systems are merely not a sustainable model. We will have to start looking outside of our own engineering craft and get interested into more complex areas of measuring and information gathering from other industries. When software increase in complexity and scale it starts to take another form, a form that more resembles a living organism than a static, mechanical machinery. Because – let's face it – the idea of trying to maintain a decoupled architecture is a nice thought and a good practice in theory to a certain extent, but guess what? At some level the different digital components will have a dependency towards each other, it is inevitable. So why not just embrace it? Embrace the fact that your architectural model is complex and it is only going to get worse?

No alt text provided for this image

If we embrace the complexity of our digital platforms we force ourselves to start considering them from another perspective; as a synthetic creature placed between the analog (mechanical) and biological (human) realms. When our simplistic mental models inherited from the mechanical world is not enough anymore, we need to adhere to a nomenclature and approaches originating from other sciences, for example the socioeconomic or even bio-science-context.

We need to look upon our system landscapes as continuously evolving synthetic organism and start testing based on this new narrative. This is when our test automation and bots are perceived by us as 'sensors'. We will start deploying a massive number of sensors in our applications, environments and business applications in order to gather all the data streams to capture what we think is relevant. What do we look for? Sometimes we know, sometimes we don´t so we need to gather this data in big data storage in order to detect and foresee correlations, patterns, dependencies and interesting trends that we can extract important insights from. In bio-science these insights are often referred to as bio-markers:

"a naturally occurring molecule, gene or characteristic by which a particular pathological or physiological process, disease, etc can be identified."

So, in our software testing approach and strategies, we need to start deploying sensors and start exploring what we might refer to as ‘technomarkers’ – signs of concerns that might lead to failures, intrusions, unexpected usage or degradation of various kinds.

No alt text provided for this image

Does these transition mean that we need a new set of tools? No – I don't think so. A sensor can be implemented using the technological toolbox we already have, for example jMeter, python or any other script-language. It is perhaps even the same test scripts we've implemented since before, it is just a matter of how we perceive the scripts and how we put them to use. We will start executing our business-critical smoke test in containers everywhere throughout our system landscape instead of manual or deployment pipeline-triggered scripts.

When we start getting used to collecting data streams together and look for various interesting patterns I would like to refer to this not as software testing, but rather Development Intelligence. We will then start experimenting even further with machine learning in our quest for knowledge and it will develop into more advanced forms of artificial intelligence. Enter hop 4. This is when AI matures and our system landscape starts to get injected with a lot of self-learning and autonomous bots. When we hit this stage in our development we will get aware of the fact that we have developed a system landscape that is so complex that we don´t know how it is set up any more and it is developing – evolving – itself.

At this stage it is good that we developed our own abilities within Emotional Intelligence, because we now need to be able to express models for ethics, morale, ability to learn, cognitive intelligence and definitions of responsibility and accountability. And – we need to be able to express it in code and teach our systems these abilities. How do we test a synthetic organisms' ability to be ethical? What is the true difference between accountability and responsibility? Working with QA in this era will surely resemble how to be a parent or a pre-school teacher that constantly make sure the systems learn new things but provides them with opportunities to fail in controlled surroundings.

Hop 5 –Test forecasting. At this stage I think the term we know as ´Software Testing´ is made extinct. Somewhere in this course of events testing has become totally submerged into developing. We've created sensors that offers data streams in order for us to understand complex events in our system landscape. Then we started to feed these data streams to our development intelligence robots to take corrective actions for themselves. When these bots indicate a significant technomarker or any interesting events in our system landscape and this event makes our robot instruct another robot to learn something new based on this event and this new experience is implemented by another robot that creates another significant event and this happens during a couple of milliseconds, you see where this is going? Humans won´t be able to keep up. End users will be able to use digital tools that will look slightly different each day based on last days usage, the system will analyze the current usage and business metrics and continuously adapt itself in small iterative manner.

How do we prepare for this transformative journey as developers and testers?

Once we've realized that software development and testing won't be the same in the future, we must go out and get interested into how we measure and gather insights from various other sciences. How do we capture brainwaves? How do we track neuron signals through our synapses? How is our central nerve system built up and how can we track, monitor and find early indications that there is something wrong? What is the latest innovative discovery within sensor industry? How do we extract data from such sensors? I strongly believe these are the questions that prepare you and your organisations for the testing industry 4.0.



I´ve gathered lots of inspiration from In Persuit of Digital Happiness Report series:

First time I heard about technomarkers were from Anders Källström from the Sustainability Circle - read their manifest here:

Testing in the digital age - by Rik Marselis, Tom van de Ven, Humayun Shaukat

  • fredrik scheja
    Fredrik Scheja
    Test Architect/Test Automation Expert / Podcaster
    070-345 27 13