BUILDING A QUALITY SYSTEM FOR SOFTWARE

Table of contents

6. BUILDING A QUALITY SYSTEM FOR SOFTWARE

What is a quality system?

Quality system: The organizational structure, responsibilities, procedures, processes and resources for implementing quality management.

Quality management: That aspect of the overall management function that determines and implements the quality policy.

Quality: The totality of features and characteristics of a product or service that bear on its ability to satisfy stated and implied needs.

The definitions above were taken from ISO 8402-1986 "Quality - Vocabulary". The definitions seem to say that the quality system implements the implementation of the quality policy. Did anyone follow that?

Let's take it in English instead. A quality system is the organizational structure etc, which controls and influences the quality of a supplier's products and services. Quality is what makes your customer happy.

Virtually everything in a software development organization influences quality, so that in practice the quality system in a software developing organization is the means for managing the software development. Some examples of what may be parts of a quality system for software:

Schedule and agenda for executive meetings

Assignments of authorities and responsibilities in the company

Procedures for project management

Templates for documents

Procedures for reviews and tests

Procedures for handling of customer complaints

Records of employee training

Procedure for internal audits

Procedures for handling of changes to specifications and programs

The central product library for software.

What is then a sufficient quality system to fulfill ISO 9001? This is a question I frequently used to hear. Managers would ask me: "If we do like this, would that satisfy ISO 9001?" "Well," I would answer, "it depends. It is difficult to say... I would have to know more about your company." I felt very uncomfortable with this kind of answer. I ought to be able to say yes or no, oughtn't I? But after a while, I realized that the answer was exactly right: "It depends". What is right for one organization does not have to be right for the another.

The basic requirement on a quality system is that it works. If an external auditor can see that your quality system works, he/she would probably only be able to find minor things to criticize.

Do we need a quality system for software?

Some of the most successful software ever marketed was created without the tiniest pretense of a quality system. This is how you do it: Take three bright programmers fresh out of college. Put them in a garage in California and wait. The kids then create all kinds of fantastic ideas and program them mostly in assembler "on the bare metal". Then you sell the programs and get very rich.

So why bother with a quality system? Well, there may be a few reasons:

Your customers may require that your software works.

You may want to modify the software.

You may have to convince your customer in beforehand that you will be able to deliver a suitable product.

You may have to hire programmers who have families and private lives, and who are not prepared to work day and night for three years.

The original whiz kids may quit and start a business of their own.

You may even have product liabilities.

If you want to know what you'll get and when you'll get it, you better have a quality system of sorts. Otherwise, why not try the garage in California?

An intelligent application of the requirements in ISO 9001 will make a software supplier better. The operative word here being "intelligent". For an informally managed organization to lift itself to ISO 9001 standard must be done carefully. There is much that can go wrong.

Building the quality system

So, let's assume that you are in a business where it matters whether software products are reliable, arrive on time and can be modified. Let's further assume that management in some way has come to the conclusion that it is important that the company be up to the ISO 9001 standard. You have been assigned the task to make this happen, so you better find out how to do it.

Who should do it?

Don't start a project or task force of low-level staff with orders to create and establish a quality system. They will eventually fail, since what they create is wrong, and it will not be accepted by the organization.

To create, establish and maintain a quality system is a task for the line management. In my view, issuing directives, assigning authority and responsibility etc is the most important of part a line managers job. This is not always understood by the managers; I sometimes see managers who happily spend their time making promises to customers, hiring people, twisting the arms of project leaders, jumping into firefighting top-priority task forces. While instead, they ought to create a way for their subordinates to work, which would make the firefighting and armtwisting unnecessary.

So, it's the line managers who shall build the quality system. Of course, though, they may use subordinates to help, but each manager should take responsibility for the specific part of the quality system which is unique for his or her organization. Managers should also cooperate around that part of the quality system which is above them.

This is also a test of the most important success factor for quality system building: Management commitment. If the managers on different levels do not feel that the quality system is important and right, it will never be built.

The most successful approach I have seen was this: An active and very persuasive quality manager planned the work and coached the line managers. The quality manager prepared and supported the line managers' work by preparing example documents and suggesting actions. He bullied the managers to spend nights and weekends working out how they actually wanted to run the company. When they knew, they could bring in other staff to help with the details.

When the quality manager felt that the company's quality system was in operation, he brought in two external consultants to conduct an internal quality audit to ISO 9001. The consultants found a few things to correct and amend. After this was done, the company was certified to ISO 9001 without any non-conformance noted.

Step 1: responsibility, authority and interfaces

When management is committed to building a quality system, I suggest that the next step is to iron out the responsibilities, authorities and interfaces. Probably, your organization is somewhat informally defined. Usually, one finds some responsibilities defined, but more rarely authorities. There are a few questions to find answers to:

Who keeps a look on X and takes action when needed?

Who decides about Y?

How is the formal relation between organization X and organization Y? Which side is in charge of what? What items are delivered between the organizations?

When all such questions are answered, you will actually know your company. Then you can continue looking at more detailed procedures for the actual day-do-day operations.

When at last management knows all about responsibilities, authorities and interfaces, this knowledge is documented and disseminated to those concerned. A person who has a responsibility and/or authority must know it, and must be able to find where it is documented.

Step 2: different processes

In step 2, each different part of the company works out its procedures under the supervision of its management.

Normally, few procedures must be invented. Your company was working before you heard about ISO 9001, so there must exist procedures, although not all are documented. Document existing procedures and perhaps polish them a little. Establish new procedures where there is a need. Resist the temptation to throw the old procedures away and invent everything anew. If you do that, you may disrupt the operation of the company, and you will delay the quality system considerably. If it worked before, then it is probably OK with ISO 9001. If you are sure it should be changed, try to do this modification as a separate activity after the introduction of the quality system. The most likely area where you will have to add procedures and rules is with regard to records. Software developers rarely recognize the need to keep records; they remember everything important anyhow, and they do not recognize the customer's legitimate need to see records.

Executive Management

Document the means for executive management to receive feedback from the operations of the company. For example, a management review of the quality system shall be held regularly. This can be done by having the quality manager once a year present for review the results from and actions on internal audits and problem analyses. The fulfillment of quality goals should also be reviewed.

Development

Document how projects are managed, supervised and planned. Prepare a template for project plans. Document the project manager's authority to prescribe project-specific procedures.

Document a basic development model with activities, document types, reviews and tests. Prepare templates for documents and procedures for reviews and tests. Include in the procedures rules for what records shall be prepared and kept.

Document procedures for identification, archiving and change control for documents and programs. Document procedures for error reporting during development.

Document procedures for delivery and installation. Document procedures for archiving, error correction and enhancements after delivery.

A tempting shortcut would be to decide that each new project shall define its own procedures and standards and document them in the development and quality plans. Then we don't need to do very much, do we? The problem of ISO 9001 compliance suddenly falls into the laps of the project managers.

First of all, this would not be efficient; having each project manager reinvent old wheels. Secondly, the project managers would in practice use procedures and standards from old project, so there would be an informal common quality system. Thirdly, if you are to get a certificate of compliance to ISO 9001, that certificate will be about your company, not any individual project.

For an auditor, it is a question of judgement how much you might be allowed to leave to individual projects. Note, however, that if two projects are using the same practices, procedures or rules, then the practices, procedures and rules constitute something common which shall be documented outside the projects.

Quality assurance

Document plans and procedures for internal quality audits.

Document procedures for collection of statistics, problem reports and other records, and the subsequent analyses to find weaknesses in work processes. Document how actions are initiated and recorded.

Sales

The sales department should document the sales process, and especially the contract review. Perhaps there are customer satisfaction investigations or reception of customer complaints. Document procedures.

You might notice that ISO 9001 is not about sales. The title does not mention sales, and neither is there any mention in the text of the standard. Still, an auditor will by inference require that activities supporting design, development, production, installation and servicing be under proper control.

Personnel

Usually, the personnel department keeps employee files. This may be a good place to keep training records also. Personnel may also be in charge of enforcing regular discussions between managers and subordinates about training needs, and subsequent updating of training records. Document procedures for this.

Also procedures for recruitment and introduction of new employees should be documented.

Switching over

Usually, it is best if a quality system can be introduced step by step over a period of time.

If the new quality system is introduced through an abrupt "switchover", or if your projects strech over a long period of time, there is the question of what to do with ongoing projects. Presumably, the old projects do not follow quality systems which are up to ISO 9001. It is important to notice that your company can not claim to have a quality system in accordance with ISO 9001, if there are projects running, which do not fulfill the requirements in the standard.

On the other hand, the old projects do not have to follow the new quality system. A good way to handle this situation is to modify the plans for the old projects, so that the rest of the projects are conducted in a manner which satisfies the standard. There is, of course, no need to redo the parts of the projects which are already finished.

But what will the programmers say?

Myth: "Programmers are creative individuals who can't be managed."

I sometimes hear managers voice concern regarding their programmers' aversion to every kind of management, rules and paperwork. "We have tried. We had a consultant prepare a comprehensive program development handbook in three volumes, but we were unable to make the programmers use it."

I have several times worked for the customer side with contracts including software development. When I have been obliged to criticize the supplier for lack of management and procedures for software development, every time a programmer has taken me aside to tell me: "It was really good that you said that. We have been pestering management for several years now, saying that we must establish rules and standards for our software development."

A number of times, I have been involved in the introduction of formalized development processes for software, with rules and paperwork. I have never heard a programmer say "What a bloody nuisance this is, paper and rules will curtail our creativity and prevent us from doing our job", or anything else in that vein.

The programmers will be no problem at all. Rather, they will help enthusiastically with the preparation of specification templates, programming rules and reviewing procedures. Under two conditions, though:

The templates, rules, procedures etc are introduced with the aim to support the programmers and help them do a still better job.

The templates, rules, procedures etc are the right ones.

For a professional to be forced to follow a rule which does not fill any reasonable purpose, is both insulting and demoralizing. Even worse is the not uncommon case, when the rule incurs extra cost at the same time as it forces the programmer to make a bad job.

It is important that it is the right persons who prepare instructions for software development. Often, a software organization has one or two "gurus", to whom everyone will go when they have problems with specifications or programs. Let those "gurus" prepare the rules and procedures. Do not leave this work to persons who are not software experts themselves. Do not use outside consultants who want to do the whole job.

The certification process

Preparing certification

So, now your company has a quality system, which is up to ISO 9001 (you believe). Why not get a third party certificate to that effect? Actually, it might be a good idea to decide about the certification when starting the work with the quality system. In that way, everyone would see a specific goal for the work, and it is also easier to get things going, if you can refer to a "foreign power", the certification body, which will eventually come to inspect the result.

In the following I will give a picture of how certification of a software developing company might happen. Remember that I am using international terminology. In USA, "registration" is used for "certification", and certification bodies are called "registrars".

Some companies are in a great hurry to get a certificate immediately after they have built the quality system, often because the manager has set up as a goal certification by a certain date. They get disappointed. After you have put a quality system in operation, you have to wait some time before attempting certification. This is because certification bodies do not certify "good intentions". You must show that your procedures are in operation, for example by having had a couple of projects applying them from project start to project finish.

How long will it take to build and implement a quality system for software development? This of course depends on your situation when you start, and on the length of your projects. If you only run long projects, you might take on a short project only to be able to show the certifiers your ability to apply the complete quality system. In general you might be wise to allow one or two years time for building the quality system and collecting evidence of your ability to apply all parts of it.

The first step is to choose a certification body. A list of certification bodies (registrars) accredited by the US accreditation authority RAB is available from

Registrar Accreditation Board
611 East Wisconsin Avenue
P.O. Box 3005
Milwaukee, WI 53201-3005 USA
Phone 414-272 8575
Fax 414-765 8661

This list does not indicate which of the certification bodies are prepared to take on the certification of software development. In other countries, go to the national accreditation authorities for similar lists.

The certification body does not have to be accredited by RAB. For example, TickIT certifications are not accredited by RAB, so if you want a TickIT certificate, you will have to go for a certificate under accreditation by the British accreditation authority NACCB. A list of accredited TickIT certification bodies is available from

NACCB
Audley House
13 Palace Street
London SW1E 5HS
England
Phone +44 171-233 7111
Fax +44 171-233 5115

Elicit quotations from a couple of certification companies. As basis for the quotation, they will require some information about your company, e.g.:

Size

Number of sites

Scope of your business

Choose certification body with the following criteria among others:

Price

References from other software developing customers of the certification body

The documented software competence of the certification body's auditors.

Insist that your software development be certified only by auditors who themselves have a thorough experience of developing software. Registered TickIT auditors are preferable.

Many European certification bodies would offer as a service a "pre-certification": A brief audit to ascertain whether your company has any major deficiencies with regards to ISO 9001. Don't use this service if you don't have to. Certification bodies are not allowed to give advice, so the result of such a brief "pre-certification" will not be very helpful. The auditors might say for example: "Your document control is insufficient and there are weaknesses in your design control". They are not allowed to suggest what you should do about it. It is much better to find someone competent in ISO 9001 and software to do a thorough audit of your operation, and to produce a comprehensive report with detailed advice as to what you should do. The reason why certification bodies are not allowed to give advice, is that the auditors would not be neutral to a quality system prepared in accordance with their advice.

When you have contracted a suitable certification body and agreed about the time for the certification audit, inform your staff clearly about the certification itself and about what is required from them. Most of us are unused to having strangers coming to us and start asking intricate questions about our way of work. It is easy to become intimidated in such situations, if one is not prepared for the experience, and it is also unnecessary. One of the advantages of having someone from the outside make an audit as preparation for certification, is that this tends to make people more used to the idea, so that at the certification audit they are able to show their best side. Tell people not to lie or hide anything, but tell them to be careful with what they tell the auditors. If one lets oneself go, the connection between brain and mouth may overload, resulting in an unnecessarily bad impression.

If people appear really terrified at the prospect of a certification audit, it may be a good idea to educate them. Describe what an audit is about, how it is done, and how to behave as an auditee. Perhaps run an audit interview in front of an audience so that people get a feel for it.

You should also tell your staff about the contents and implications of ISO 9001, but don't send them all to courses about ISO 9001. The only ones who have to know ISO 9001 are the managers and others involved in the design of the quality system. They create a quality system which fulfills the standard, and the staff only have to see that quality system, not the standard itself.

Don't choose a time for the certification

when you just have had a reorganization,

when half the programmers are away on a course, or

when you are in a critical phase in a project

The certification audit

When the time and duration for the certification audit has been agreed, the certification body will prepare an audit program, detailing what department and what type of staff they want to meet and when. The following functions will surely be in the program:

The managing director

the quality manager

all department heads

personnel, training

project managers and software engineers in ongoing projects.

The first day of the certification audit, there will be some trepidation on the part of your staff. It is "graduation" after a long time of preparations. On the positive side there will be curiosity and eagerness to show one's work and ability. On the negative side, there will be nervousness and a feeling that the certification body is trespassing.

The first day starts with an entry meeting, where the certification auditors meet with company management and with the persons who will act as guides during the day. At the entry meeting, the lead auditor will explain the purpose of the audit and present the auditors. Your representatives will introduce themselves. The audit program will be discussed and possibly changed. An office will be put to the auditors' disposition for internal meetings. An overview of the company will be presented, possibly followed by a brief tour of the premises.

Then, the audit proper starts. The goal for the audit is to ascertain that your quality system is up to ISO 9001, and that it is implemented in the day-to-day operations in your company. The auditors will split up, each with a guide, and go out to interview different persons. The interviews will normally take place at the interviewed persons place of work. The auditor will be interested in how that person works, and what documents are available. A good auditor will not start asking questions from the standard or from a checklist. Rather, the auditor wants the interviewed to tell with his/her own words about what he/she is doing. After that, the auditor will ask a few questions. With a manager, he/she will ask for example:

"How do you manage your organization?"

"How do you get feedback from your organization?"

"What records are there from your control?"

When the interviewee mentions a document, the auditor will say: "Show me!" This is a conditioned reflex of all quality auditors. To listen to people telling you about their work is interesting, but it gets substance only when you can inspect something tangible, for example a specification, a program or a review record.

At the end of the day, the auditors will meet to compare notes, and afterwards there will be a meeting between the auditors and the company managers, where the auditors will present any non-conformances discovered during the day. You will then have the opportunity to correct the non-conformances and have the auditors clear them during the audit. At least, you will be required to produce an action plan for the non-conformances.

Some non-conformances can not be corrected immediately, for example if some large activities have not been conducted, such as internal audits or design reviews. Also, even if you are able to correct all non-conformances concerning, say, document control, the auditors may still raise a general non-conformance regarding this quality element. The auditors only do sampling. If they find many non-conformances in a certain area through sampling, they will assume that there are many undetected non-conformances for every one they uncover. It is then not enough to fix the problems the auditors happened to find. You will have to go through the sampled area thoroughly to ensure that there are no non-conformances left, which could be found in another sampling.

Each morning there will be an "entry meeting", when you will have the opportunity to present corrections and/or action plans for non-conformances.

So comes the last day of the audit, and in the afternoon, there is an exit meeting planned. At this meeting, the auditors will tell you the outcome of the audit. There are three possibilities:

The auditors will recommend immediate certification. In this case, there must not be any unresolved non-conformances. This is a rare case, but I have seen it happen.

The auditors will recommend certification pending the satisfactory resolution of all remaining non-conformances in a certain time-limit, for example three months. The certification body must be able to confirm the fact that the non-conformances have been corrected, either from documents received from the company, or through a visit by an auditor.

The auditors can not recommend certification. You will have to work on your quality system and its implementation in the organization, and then apply for certification anew.

Hopefully, your case will be 1 or 2 above, so that you will then after some time receive an impressive certificate saying that just your company is now a member of enviable group of software suppliers, who fulfill the requirements of the international quality standard ISO 9001.

Maintaining a certificate

The certificate you just received and put up on the board-room wall is valid for a certain period of time, usually three years. After that time, you will have to apply for certification again and do it all over again. Hopefully, the process will be much simpler that time.

So, when you have received your certificate, you can lean back and relax for three years. How wonderful after all the overtime you put in to get the quality system going.

Forget it; the auditors will soon be back. In your contract with the certification body, it is stated that they will do regular audits twice a year during the time of the certificate. Often, an organization is kept on its toes until the certification is done, but then the reaction comes, and the disciplined way of work is replaced with the usual sloppiness. That's why the certification body will be back to check that your company is not one of these organizations. These regular surveillance audits are less comprehensive than the certification audit, and usually the auditors concentrate on different areas of your quality system each time.

During the time of the certificate, you can still loose it. In some situations, the certification body has the obligation or authority to withdraw your certificate. Examples are:

If you fail to correct in prescribed time a non-conformance, which has been found in a surveillance audit.

If you use the certificate improperly in your marketing, for example indicating that this is a certificate of the quality of your products.

If you don't pay the bills from the certification body.

A major reorganization or a merger with another company can make the certificate invalid, but often this can be avoided if the certification body is kept involved in the change, so that its auditors can audit the new organization and confirm the certificate.

Maintaining a quality system

A quality system which is not maintained will quickly degenerate. Responsibilities and authorities will change as people move around in the organization, rules will become obsolete because better ways have been found, etc. Of course, the certification body returns twice a year to check up on you, but they only do sampling, and your are not allowed to use their audits as a means for finding out what has to be done; you must find out for yourself.

You have four main tools for maintaining the quality system:

Internal quality audits. Aside from being "police actions", making sure that people adhere to the quality system, your internal quality audits shall be aimed at finding out if something is wrong with the system itself and looking for opportunities to improve it. For example, the fact that someone is not following a procedure may be an indication that the procedure should be changed.

Your continuing improvement activities. ISO 9001 para 4.14 requires that you actively collect and analyze information about problems and shortcomings in order to find possible improvements. The standard further requires that you have means for implementing and following up decisions from this activity.

Suggestions and complaints from your staff using the quality system. Make clear to everyone, that suggestions and complaints are welcome; make it easy to document suggestions and complaints (on the system or using a form) and make sure that people know where to go with them. The suggestions and complaints can the be input into the process at 2) above.

Post mortem analysis. It is a good idea to sit down immediately after the finish of a project and discuss what was good and what wasn't so good with the way the project was run, with rules and procedures etc. Write a short report and enter it into 2) above.

Remember: If you don't improve, then you deteriorate. To stay on top, you must improve all the time.