DevOps is neat, I like it, I find it natural extension to Agile movement and timely progression of it. Of late while consulting and coaching for the clients it seems to have entrenched me, to a point where it made me see every aspect of software development from the vantage point of (through?) DevOps. It’s all good but it’s also good to introspect your thoughts and observations, I did that. And I realised that for every team I worked with, the common recommendation was to bring the testing team upto speed or it was me saying DevOps is not working for you guys because Testing is lacking the edge or something to that effect, each and every time. Did I focus too much on Developers and Ops team to make them work in synch, collaborate and left testing to focus on automation and making continuous testing work? And hoping that this would make the testing team develop the necessary chops to deliver in DevOps projects. I was wrong, cutting it short, here is my conclusion – You can make testing effectively in DevOps only when the team has mastered the art of Agile testing. Just doing test automation, integrate testing tools with DevOps tools, doing continuous testing will not help much if the Agile fundamentals are lacking in testing.

So I take a step back and putting together my thoughts (without being too prescriptive) on actions to take to become agile in testing. Mind you I have kept it grounded to remain within the domain of Agile QA only and not ventured into Continuous Testing or Testing in DevOps space… back to the basics team (documenting my thoughts at random so you may find the sequencing not in ideal order). Here it goes:

Agile is iterative so is the testing: In any Agile model the requirement gathering and documentation, and design activities, and development activities, are iterative in nature. So must testing activities. If Testing team is not adapting to this then it will lead to failure.

Testing activities start with requirement phase: Requirements are ever evolving and there will not be a time where comprehensive requirements specification will be made available, therefore testing strategies can not rely on complete specifications. In fact work with rest of the team during the requirements gathering and grooming activities and create test strategies on the go.

Team formation and “T” shaped testers skills – This expectation has been around for long however the practical problem faced is that team may not even know what additional skills are needed and to what extent. This needs to worked upon and gradually skills need to be introduced to the testers.

Embedding Testers into Development teams for story testing including the unit tests.

For releases and niche testing it is good idea to have independent testing teams (and even have test hardening sprints)

Test environment and test data setup – This needs to be handled upfront as requirement and technical task while requirements gathering.

Test Automation and regression testing: Test automation strategy needs to be planned in sprint zero itself and have automation done in every sprint. Planning to do big band test automation after several sprints does not work and leads to high risk of having ineffective regression pack.

Perform static analysis: Static analysis to be used to check the quality where an automated tool checks for defects in the code, often looking for types of problems such as security defects or coding style issues.

Lead finalising the Acceptance Criteria for Business User Stories: Combining Business Requirement driven approach to testing by using Behavior Driven Development (BDD) using Cucumber. The greatest challenge with adopting BDD could be the lack of skills amongst existing requirements practitioners and testers. This is yet another reason to promote T-Shaped skills within the organization versus narrowly focused specialists.

Utilize the code coverage tools to ensure that the automation tests are offering sufficient coverage on code. Not doing this will lead to pesticide paradox and “false-pass” tests.

Plan and stay with the Development cycle: Do all that can be done but do not get one (or more sprint) behind the development cycle. Always plan for n-1, n and n+1 and it becomes essential that the testing team participants in sprint and release planning sessions.

Defect Management: In Agile the defects are just another type of requirement. A defect can be documented into the requirement to fix th defect against the Story X. In such situations the customer typically needs to pay for new requirements that weren’t agreed to at the beginning of the project but should not pay for fixing defects. This leads to commitment for zero-defect goals and merging the requirements and defect management processes into a single, simple task board gives a wonderful opportunity for process improvement and defect prioritisation.

Another practice that works wonders for quality is All-hands demo: Although there might be several project stakeholders working directly with the team we could have several hundred (or thousand) of actual users that don’t know what’s going on. Thats why I advocate an “all-hands” demo/review early in the delivery life cycle, typically two to three sprints into the construction phase, to a much wider audience than just the stakeholder(s) versus the few with whom we are working.

Going by Agile-Lean principles, testers should look for defect prevention and not just focus on finding defects. By this the definition of defect from lean perspective is:

– Building something with low quality (having bugs, traditional testing perspective)

– Building something the customer did not ask for because a requirement was misunderstood (BDD can help via executable requirements)

– Building something the customer did ask for, but later realized it was not what they meant and they do not want it now (BDD can help, also frequent all-hands demo)

Metrics:

Go beyond the typical Agile metrics such as Burn-Up and Bund-down charts, velocity and story points tracking. Some metrics that can lead to high quality:

• Running Tested Features
• Business Value burn-up
• Technical Debt
• Code Metrics – Cyclomatic complexity, Coding standards violations, Code duplication, Code coverage, Dead code, Code dependencies, Abstractness

Metrics are merely indicators and trends, they should not lead to blaming individuals in the team. They helps us get the desired Agile “inspect and adapt” feedback loop and we don’t collect data just for the sake of collecting data.

Hope the above helps and when in doubt check the Agile Manifesto and the guiding Agile Principles .