Table of contents
5. COMPARISONS WITH OTHER SCHEMES
The SEI capability maturity model (CMM)
CMM was developed at the Software Engineering Institute in Pittsburgh, and it is very much a rival to ISO 9001 for software.
CMM is a scheme to classify a software developing organization according to its capability. CMM identifies five different maturity levels for software developing organizations. The levels are as follows:
- Initial. The software development is run informally, and depends on the competence of some persons.
- Repeatable. There is a common system for project management and control.
- Defined. There is a common system for the software engineering activities.
- Managed. The software development process is stable and giving a consistent product quality. Measurements are used to keep the process and product under control.
- Optimizing. The software development process contains its own improvement process.
There are 4 main differences between ISO 9001 and CMM:
- ISO 9001 is intended for most industry, while CMM is software-specific.
- CMM is more detailed and specific.
- ISO 9001 establishes one acceptable level of a supplier's management and processes, while CMM is a tool for assessing the supplier's software ability on a scale from one to five.
- ISO 9001 focusses on a customer - supplier relation; CMM is mainly concerned with the software development process as such.
Several papers have been published in recent years, discussing the relation between ISO 9001 and CMM (see references). It appears that an organization which is certified to ISO 9001 would achieve most requirements for CMM level 2, and an organization on CMM level 2 would fulfill most requirements in ISO 9001. There are some requirements in ISO 9001 which are not well covered in any level of CMM. For example, the requirements in ISO 9001 which refer to a customer (e.g. 4.3 "Contract review" and the part in 4.14 "Corrective and preventive action" about customer complaints) are not very well covered in CMM, since customers are not conspicuous in CMM's perspective.
Should a software company go for ISO 9001 or CMM? To go for CMM level 2 would be easier in some respects, since CMM is software-specific. The company could also later go for higher levels of productivity and quality. The advantage of ISO 9001 is that the standard is very well known and accepted in different industry branches around the world. A certificate to ISO 9001 is probably more useful than a certificate to a certain CMM level.
The first criterion for choosing between ISO 9001 and CMM is of course the need. If you need CMM, for example to be able to do business with US DoD, go for CMM; if, on the other hand, you need an ISO 9001 certificate to do international business, go for that. However, if both are needed, or if there is not a specific need for a documented CMM level, I would recommend that a software organization first goes for certification to ISO 9001 according to TickIT, and thereafter uses CMM to improve further.
Bamford RC, Deibler WJ: Comparing, contrasting ISO 9001 and the SEI capability maturity model. IEEE Computer, October 1993.
Paulk MC, Bamford RC, Deibler WJ: Basis of contrast between ISO 9001 and SEI Capability Maturity Model challenged. IEEE Computer, February 1994.
Paulk MC: Comparing ISO 9001 and the Capability Maturity Model for Software. Software Quality Journal 2, 245-256 (1993).
Paulk MC: How ISO 9001 compares with the CMM. IEEE Software, January 1995.
F Coallier: How ISO 9001 fits into the software world. IEEE Software, January 1994.
IEEE (Institute of Electrical and Electronics Engineers) publishes a number of software engineering standards. IEEE 730 is titled "IEEE Standard for Softwarere Quality Assurance Plans" and was last revised in 1989. As the title implies, the standard is about a "Software Quality Plan". When stating the requirements on the contents of the plan, the standard also puts requirements on the management of the software development. IEEE 730 states minimum requirements on the documentation to be produced, and on the reviews and audits to be held. "Audit" is in this context something different from the "quality audit" we have mentioned in this book.
IEEE 730 covers approximately paragraph 4.4 "Design control" in ISO 9001.
AQAP-110 and AQAP-150
The military purchase all kinds of equipment. Therefore, NATO publishes standards intended for use by the customer side in large acquisitions, two of them being AQAP-110 and AQAP-150. "AQAP" is an acronym for "Allied Quality Assurance Publication".
AQAP-110 edition 2 was published in February 1995, and is titled: "NATO Quality Assurance Requirements for Design, Development and Production. It is basically ISO 9001 with some additions. AQAP-110 has the same structure of its requirements as has ISO 9001, and mostly, AQAP-110 only says: "ISO requirements apply." AQAP-110 replaces the older AQAP-1 standard, and like it, it contains no software-specific requirements.
AQAP-150 edition 1 was published in March 1993, and a draft edition 2 came in September 1997. Its title is: "NATO Quality Assurance Requirements for Software Development". AQAP-150 replaces the older AQAP-13. It is intended for use either together with AQAP-110 or on its own in acquisitions including software development. A draft version
Since AQAP-150 is software-specific, we should perhaps compare it to ISO 9000-3. The first difference is that AQAP-150 is a standard, while ISO 9000-3 is a guideline. ISO 9000-3 says "should", while AQAP-150 says "shall". AQAP-150 is strongly focussed on project-specific issues, and in essence it contains requirements on the contents of a "Software Quality Assurance Plan" and on the activities controlled by the plan. AQAP-150 is vague on such things as quality policy, management review, contract review, internal quality audits and training. Also, it says very little about maintenance. Apart from this, AQAP-150 differs from ISO 9000-3 only in details.
In 1995 ISO and IEC (the International Electrotechnical Commission) jointly published the standard ISO/IEC 12207 "Information technology Software life cycle processes". Together with six additional annexes, ISO/IEC 12207 was published in 1998 as IEEE/EIA 12207.0. This standard has superseeded MIL-STD-498, which was previously used by the military.
ISO/IEC 12207 contains descriptions of processes, activities and tasks involved in the acquisition, supply, operation and maintenance of systems and software products. The text is formulated as requirements, which can be tailored by deletion.
ISO/IEC 12207 covers ISO 9001 paragraphs 4.4 "Design control", 4.5 "Document and data control", 4.6 "Purchasing", 4.14 "Corrective and preventive action", 4.15 "Handling, storage, packaging, preservation and delivery", and 4.17 "Internal quality audits" in more than 50 pages. It also covers areas outside ISO 9001.
One problem with the use of this kind of standard is when it is imposed on a supplier who has not used it previously. If the supplier already has a suitable development model in operation, it should be used. It is not to anyone's advantage to force the change to the ISO/IEC 12207model for one contract only. It takes time to change the way of working; the standard is not always easy to understand; and there may be misunderstandings when attempting to use it for the first time.
The requirements in ISO/IEC 12207 are intended to be tailored by the customer before imposing them on a supplier. This is especially important for small and not-so-large projects, where the standard easily become oppressive otherwise.
A good way to use ISO/IEC 12207 is for the customer include in the contract pointers to specific requirements in the standard, which the supplier must fulfill.