In formulating my originally ambivalent views on the subject, I had the great pleasure of meeting and discussing the new standard with Henrik Andersson from the International Society for Software Testing. This is a testing association closely linked to the Context-Driven School, noted for their open letter to the ISO in protest over the implementation of the ISO 29119 standards. The letter eloquently put the idea of standardisation, the expertise and bias of the ISO panel and the voluntary nature of its implementation into question. It is well worth a read as a summary of the key points of contention.
If there is one point of concern I have, it is that I have no real idea what problem the standards are meant to solve. In my time in the industry, I have found that companies and (good) testing teams generally apply a selection of testing techniques and methodologies to their work in a way that achieves results for them. The idea of an off-the-shelf standard being applied to projects and life-cycles of different types, lengths, resources and requirements is extremely unlikely. It is also unlikely that the testing community as a whole will accept this.
It could easily be said that the standard is purely voluntary and testing teams would be allowed to pick and choose aspects of each standard that appeal to them and discard the rest, but if that was the case then the need for an overarching standard at all is essentially moot and adds nothing of value to the industry.
The IT industry, due to its immense variability and the wide range of products, stacks, methodologies, projects and technologies that make it up, does not fit easily to the idea of standardisation. While some areas such as Project Management have been partly amenable to methodologies such as Prince 2, others such as software development and database administration have not. It would be impossible to impose, voluntarily or coercively, a standard all developers and tech leads would agree to.
In software testing, the divergence of ideas and methods is, if anything, even greater and less amenable to reconciliation. Cem Kaner in his comment to the ISO 29119 petition made the point that we as an industry cannot even agree on what software testing is, or what its basic practices are. We have little hope on unity on its more specific approaches - especially in the enormous detail described in the five parts of the standard. Efforts to enforce standardisation from the top down could be seen to turn testing into a set of rote processes to tick off instead of an intelligent multi-part study and learning experience regarding the state, performance and suitability (among lots of other properties) of a piece of software or web application or system in a specific context. The only way to implement adherence to the standard everywhere would almost certainly be through either coercion or the sidelining of those who speak out against it.
So who in the testing industry supports the new standard? Presumably the ISO will state that the Context Driven School is a vocal minority and that the majority of so-called "Factory" testers will adhere to the new standard. Interestingly there is little reaction from testers in support of the ISO changes I have read online - certainly not compared to its detractors. I don't have the evidence to support this but I would not be surprised if the majority of non-CDT testers see the ISTQB mostly as an accreditation hurdle to jump through for resume padding and the ISO/IEEE standards as guidelines that they rote learn during the exam. They will then naturally implement whatever processes and techniques they deem necessary for the project they are on instead of keenly box-ticking the standard. That is hardly a groundswell of support.
My personal view is that since the ISO does not have the support of the entire testing body and can do nothing to coerce testers into implementing it, there is a good chance that the standards will be largely ignored by most of us and the ISO will be seen as irrelevent. This is unless the large testing consultancies use adherence to the standard as a form of marketing and force their consultants to use it in its entirety, regardless of the context, which would be disastrous.
Other points I have in protest against the standard, themselves iterated in previous comments, I have outlined below -
- I feel such an overarching standard detracts from testers and test managers’ choice to decide on the best practice necessary in the context of their environments and projects, especially if it is deemed compulsory over time.
- The sheer amount of setup and documentation required to adhere to the standard could shift a sizeable amount of the workload of testers from actual testing to writing enormous documentation that may be superfluous. This was also a notable criticism of IEEE-829.
- If anything it could be suited to large waterfall SDLC projects but seems to be at odds with more modern agile development approaches which emphasize smaller, iterative test cycles that emphasise teamwork and man-to-man communication over process and rigid documentation. In this way, it seems dated - as if the ISO haven’t kept up with wider developments in the industry.
- The testing community in general is not behind it and the consultation process seems to have been limited to a handful of contributors. Since no single body can be said to speak for all testers, this is inherently likely to lead to division and failure.
