Back to blog home

Continuous delivery and the evolution of QA

Software delivery moves faster than ever, thanks to a combination of Agile practices and continuous delivery. Here I explore how this is changing the role of software testing – with some insight from TestBash Brighton 2017.
Many technology companies are striving towards the holy grail of continuous delivery (CD). For us here at Inviqa, continuous delivery means the ability to release frequently, on-demand, and with little to no friction or risk. 
Ultimately, continuous delivery is about delivering more value to our clients and their users, and being able to respond rapidly to changing business conditions. To achieve full-scale continuous delivery we need to adopt new tools (like our very own ContinuousPipe) and technical approaches (such as git branching strategies and feature flags).
But more importantly, we need to adapt the way we work individually and as a team. For the quality assurance (QA) team, this evolution is three-fold:

1. Expand your focus

Continuous delivery presents a dramatic change to QA workflows whereby end-to-end testing at the point of integration is only one of many testing strategies we can use. To ‘do’ QA within a CD context, we need to also get involved with other activities such as release management, release planning, server admin and devops, requirements analysis, performance testing, and security testing. 
Some of these are information-gathering activities while others are alternative forms of testing that help us to determine the quality and / or robustness of a particular change.

Continuous delivery projects can churn out new features and fixes at an impressive rate, so it’s important that we gain a thorough understanding of the system as a whole, as well as the implementation details of each feature or fix.

James Sheasby Thomas 

2. Test early and often

Continuous delivery demands a ‘shift-left’ approach to QA whereby testing – that is, the activity – is only one part of the quality assurance process. As well as getting involved with discovery, design, requirements gathering and estimation, Inviqa’s QAs are using our home-grown ContinuousPipe platform to test feature branches, release branches, and ‘mainline’ branches (e.g. staging, develop) in their own isolated environments. 

We can even use ContinuousPipe’s cp-remote command line tool to run commands on, or otherwise manipulate, environments, or set up our own remote development environments to play around with. 
Crucially, whenever a developer pushes a fix to a branch, the corresponding environment will rebuild in minutes – so no more waiting for fixes to be merged, or triggering manual build jobs! 
Nonetheless, tools like ContinuousPipe are relatively new, so as a QA team we need to collaborate with developers and other technical colleagues to work out how best to evolve our processes to make full use of these tools.

3. Share QA knowledge and experience

As Amy Phillips pointed out in her opening talk at TestBash Brighton 2017, we can’t be the sole gatekeepers of quality on fast-paced CD projects.

Even if we never took holidays, it would often be impossible to give every feature and fix the time and attention we’d like to. 

QA has always been a bottleneck – most teams have more developers than testers / QAs – but on CD projects, that bottleneck has the potential to become even more pronounced. 
One solution to this problem is to add more QAs to the project, but another option is to get other team members involved in your testing.

Testing is a job role, but it’s also a skill that can be taught to others fairly quickly.

James Sheasby Thomas

On my projects at Inviqa, I’ve had some success with asking developers and PMs to help me set up environments ready for testing, explore specific edge cases, and document the implementation details of a feature that’s ready for UAT. 
This is especially helpful when deadlines are tight or the tickets are piling up in the QA column, and it fits well with the collaborative nature of continuous delivery projects. More importantly, by teaching our colleagues about testing we can help to spread quality throughout our teams and the organisation as a whole. This fits in well with the ‘shift left’ theory of QA, where quality is a key component of each stage of the process.

Change your approach to testing

CD projects are often fast-paced and confusing, as Amy Phillips highlighted in her talk. As a result, she explained, it can be difficult to know what each team member is doing or understand all of the ‘parts’ that make up the system you are testing. 
Amy explained how being on a continuous delivery project forced her to change her approach to testing and the relationship with her colleagues. On such a fast-moving project she couldn’t possibly test everything, but as a testing expert, she could teach her colleagues how to test their own and each other’s work. 
The talk, alongside the wider TestBash Brighton 2017 speaker programme, was a crucial reminder of how the move towards frequent, small releases is changing the role of the tester or QA –  and that we stand to gain much by sharing our learnings and experiences as we all work through the same challenges.

Inviqa is a technology partner and consultant specialised in ecommerce, content management solutions, and digital innovation. Get in touch to discover how we can support your digital journey.


About the author

James is a QA analyst at Inviqa. He works across multiple project teams to ensure the software we build is robust and works as intended, conducting extensive functional, cross-browser / device, and integration testing. 
James also gets involved with requirements analysis and gathering throughout the project lifecycle. With a keen interest in usability and accessibility, he brings to his role a strong understanding of the complex needs of users and how they use the web. Since joining Inviqa he has worked closely with developers, PMs, AMs and clients to help facilitate a move towards continuous delivery and a truly collaborative QA process.
A regular attendee at QA conferences and community events, James had this to say about TestBash Brighton 2017:
‘Speakers and attendees were friendly and approachable, and each talk was directly relevant to my role at Inviqa. Above all, being at TestBash felt like joining a ready-made community for a day – from the pub drinks the night before to the board games at the end, it felt like I’d known my fellow attendees for years!’

Related reading