QA

Requirements Traceability

Agile and QA approaches – Requirements Traceability

Following on from my post two weeks ago about specification by example and application maturity, this piece is about requirements traceability.

Software development processes have traditionally worked from signed artefacts and signing up to agreed pieces of work. This would normally describe a set amount of development time, some testing time and some support time to resolve defects and help bed-in the new application. An important part of this process is a description of what will be delivered. This is a key document in the specification by example process. It is important when creating the original statement of requirements and describing the user stories in tools such as Speclog that all requirements have been captured and that there is a common understanding between the project sponsors and the suppliers of how this functionality will be used. By describing key business flows in terms of the behaviour of the application an opportunity is provided for areas of discussion and the understanding of the supplier and the customer can be explored.

These activities drive the test approach. Traditional testing analysis would follow on from here. Often this would be long weeks of Business Analysts creating specification documents and from there test analysts would follow the same path and develop test packs which examine and validate these requirements. In the mean-time developers will work with the documents and the outline discussions, and begin their development approach. In the old days this would have created detail specification of requirements, a detailed development approach and detailed test approach – all of which would have to be approved and signed off. This produces a lot of documents and the big problem with them is that it assumes the requirements are fully understood at the beginning of the process. Any changes are managed through change requests, which require impacting and updating of all these documents. It is not a flexible process but although slow, delivers good quality software.

In the same way that Agile has helped to remove all the unnecessary documentation in development and testing of new applications, specification by example is looking to broaden/develop this concept to address the requirements itself. In the area of Quality Assurance the big challenge with ever more complex systems is measuring what areas of the requirements are being met by which development activities and then validated by which testing activities. In its broadest sense specification by example attempts to address this by a straight-forward approach that states that tests will be developed for each requirement specified in the user story.

The challenge is what artefacts we need to create and how they fit in with the user stories. For instance can/will test approach documents be taken straight from the stories that are created in the analysis. The examples and systems that have been discussed in Agile conferences and across the web have been small with simple business processes which have limited number of functional combinations of application modules. And these systems have been created from scratch or are small improvements. This work hasn’t taken place with existing systems that may not have full documentation or where new software and changes to existing applications take place.

This is where testing theory and previous processes can supplement the given process. It is important that the relationships between the requirements and stories are understood. What are the most important part of the applications ? Which are the most complex? By using the traditional requirements traceability it is possible to create a an application relationship map – which then can be used to drive the test plan and more importantly be used to manage the delivery of the application functionality. This will help with deciding the critical path and the main release points where key modules from a testable business process.

Where defects are being seen then it can steer back to the stories and allow us to understand whether there is something missing from the requirements or something has changed or details need to be more fully understood. This takes us back to the largest flaw in current development/test methodology. When the initial analysis takes place assumptions are made using the business/test/developer analyst experience and understanding of the business. The won’t necessarily align and more importantly they will change as more work is done. It is vital that all that information is shared – and that is at the heart of collaborative processes such as specification by example try to engender. But at the moment the process seeks to implement a one-size-fits-all approach and only looks at basing everything on the user stories. There may well need to be additional process activities.

What we want to find out over the next few months is how requirements traceability works in the Specification by Example area. It has proved to be a valuable tool in Agile projects – where there isn’t the time to record all the details of all the requirements and in Waterfall where the long time periods of development and testing activity can be managed through the matrix. It has weaknesses – it doesn’t work well across a large number of conflicting requirements and doesn’t work well when the business capability being developed splits into small components. But it will be interesting to see what it produces and future blogs we will be recording what we see.

Old Globe

The Changing World of Testing

I’m part of Intechnica’s (fairly new) QA team, who are bringing in new perspectives to Intechnica, as we are from the testing world and Intechnica is traditionally a software development (as well as performance assurance) focused company. My background is not only test leadership but also test culture, architecture, working on methodologies and helping them align with the places a business is going to go.

In our world there have been big changes since the banking crash in 2008. In that time projects were large and all software testing was big budget to go with big software and big applications. But increasing pressure to make application changes faster & smarter has resulted in radical changes in how software is developed. Now applications are developed strategically and are being devolved into web services and cloud applications – Software as a Service (SaaS) – where focus is purely on the service, not how it is served.

The software engineering world is now focused on application capability, making sure requirements are clear and that they are tested as early as possible. The old job roles of test analyst, lead, and the follow on role of Quality Assurance, where people like myself would validate the work of out-sourced testing units to ensure that requirements are in line with business expectations, are melting away. Now test is being driven throughout the software life-cycle – from initial requirements to business validation. So what will these roles turn into ?

There are quite a few discussion pieces going on out there now; Some predicting that formal test and test groups will be go the same way as operations and capacity planning (what I used to do). Why would we need specific testing activity when testing is part of all project work? This article discusses this possibility: The Shrinking Role of QA

I agree that testing is changing. The question is, how do we become part of the new conversation? I think the old test/QA skill sets still remain important. How will the new software function be used? Will the expected behaviour of the application match the business opportunities outlined in the proposal? More importantly (and this is the bit that is certainly beyond current defensive method thinking right now) are there opportunities to give the users of the project a better product which brings together a couple of requirements in a more structured useful mechanism? Can we deliver exactly on time with just the requirements the customer wants instead of waiting for all the functionality to come together?

Test Analysts, Leads and Architects need to keep doing what they do best – talk through the requirements with all parties involved – work with development teams and help make sure the bigger picture of the overall solution isn’t lost – and working with implementation teams on how to get this out to the customers with the appropriate business proving trials – and, most importantly, tracking and validating the requirements, making sure any changes are done in a structured way that keeps all groups’ view of the changes aligned.

The names will change, but the role will continue to be planted at the heart of the project delivery mechanism.