Study notes on Cobit: Managing quality

What is quality?

Are your users happy with, say, your email service? This is what quality is all about. If the email system is so slow or fails to catch spam effectively, the users may be unhappy about it. If you (the CIO) don't know about it and thus aren't doing anything to address the problem, they (probably including the CEO) will lose trust in you and you will have difficulty getting budget for existing or new initiatives. So, getting the users happy about your services (quality) is critical for your career as the CIO.

How to ensure quality?

Take the email service as an example, it is basically an automated service so the quality mainly depends on the email software and the infrastructure (server and network) instead of human being. How to ensure the quality of, e.g., the email software? You may follow such a process:

1. The CIO, the business analyst and the solution architect discuss with the representative users and other stakeholders on the requirements. For example, you may determine that it needs to support shared and personal address book, anti-spam, anti-virus, web access from anywhere, digital signature, company wide archiving (for legal purpose). There are also non-functional requirements such as: it shouldn't take more than 1 minute to process and deliver an incoming mail (performance), all client communication must be secure (security), the service should be available 99.9% of the time (availability), etc.


2. The architect evaluates the email packages on the market to identity those that support the above requirements.


3. The architect selects the one that meets the requirements, at a lower cost with a lower risk.


4. The architect requests the representative users, in your actual environment with a heavy work load.


5. The CIO finalizes on the decision with all the stakeholders.


6. Deploy the solution.


7. Get user feedback.


8. Get user feedback regularly to see if the service gets deteriorated (e.g., is it getting slow due to increased volume of emails?) or there are new requirements (e.g., shared calendar?).

If you check carefully, you will note that this process is actually applicable to most types of solution acquisitions and has little with the specifics of email service. The process is basically:

1. Get requirements (control point for quality).


2. Evaluate & test possible solutions (control point for quality).


3. Select a solution.


4. Validate the solution (control point for quality).


5. Finalize & deploy on the solution.


6. Validate the solution again in production (control point for quality).


7. Monitor the quality regularly and improve as needed (control point for quality).

That is, to ensure quality, all solution acquisitions should adopt this process. Adopting such a process with quality control points is the basis of quality management. Of course, you also need people capable of doing this type of work (in this case, the business analyst and the architect).

Ensuring quality throughout the whole enterprise

The above process deals with the quality of acquired solutions, but what about others such as development of applications, support services and maybe other non-IT products or services? To ensure that everything is covered, there should be someone higher up such as the quality director who is in overall charge of quality (in ISO 9000, this is called the management representative). He should somehow ensure that the processes for the company to develop its products or to deliver its services (including those for internal use which is typically the case for IT) have incorporated control points for quality.
As this is a lot of work and much of the work requires professional knowledge (e.g., it's difficult for a non-IT person to ensure quality in software development), the quality management representative should ignite the care for quality in the minds of all staff members, so that they drive the development of the processes to ensure quality instead of them being driven by the processes.

Summary

As you can see, to ensure quality, your company needs the organization (quality management representative and all other people actively working to ensure quality and their relationships) and the processes in place designed to ensure quality. These two are the main elements of a quality management system (QMS).

Open letter to certification providers

Dear certification providers,
In recent months I've interviewed three candidates with CCNP certifications and I was very disappointed to find that two of them didn't know how a switch differs from a hub or how it learns the MAC addresses. My first reaction was my disappointment with the lack of integrity of the people and the general availability of brain dump or even real test questions. However, from a more positive view, this has stroked me to think about what we can do to help you improve the situation?
If we consider certifications as a service in the perspective of quality management, then we can clearly see there is a huge problem in it: the exam candidate is both a client and a user (and also a piece of client-provided material :-) ), but there are many other users of this service out there; they are the prospective employers, job interviewers, peers and etc. A key requirement in quality management is to measure if the users are satisfied with the service output. Obviously, as a prospective employer, I am very unhappy with the service output because the CCNP certificate is not reflecting the real expertise of the certificate holders. However, none of you are providing ways for unhappy users like me to provide feedback on your service output. Without such a feedback mechanism, I really don't see how you can ensure the quality of your service.
Therefore, I'd request that you establish a mechanism to accept feedback. For example, provide a website to let me report on those certificate holders who obviously know little about the subject matter. Then follow up with investigation, re-certify as required and revoke their certificates. Just like a digital certificate whose private key has been compromised, such revoked certificates should be published on a website like a CRL.
This is about handling an individual incident. If there are a significant number of such incidents, you should escalate to become a problem (in ITIL term). It means that you must then identify the root cause and to plug the hole to prevent similar incidents, like introducing performance-based exams, reference-based certifications and whatever it takes to fix the problem and save your reputation.