By Craig Cochran
Auditable requirements and objective evidence form the foundation of every audit. Indeed, one cannot function effectively as an auditor without a full understanding of these concepts as they work hand-in-hand to produce a complete picture of how the organization performs.
Auditable requirements
Let’s take an in-depth look at auditable requirements since I’ve already covered objective evidence here. Auditable requirements are the obligations that the organization has committed to implementing. Most of an audit is spent verifying that requirements have been met. Here is an example of a requirement: “Technicians must have the customer sign the work order after repairs have been completed.”
The requirement comes out of a company procedure. The company wrote the procedure because managers deemed this requirement (and others) to be important. We can’t see the entire procedure, but there are requirements before this one and probably some after it.
During the preparation phase of an audit, the auditor requests to receive various procedures and documents the organization has written. These are read and analyzed to determine which requirements are worth verifying during the audit. Auditors simply don’t have time to verify every single requirement. Rather, auditors choose what they believe to be the most important requirements to verify.
Inexperienced auditors generally struggle with identifying the requirements they should focus on—the auditable requirements. Here are some guidelines:
- Requirements for key production processes
- Requirements for inspecting product/service
- Requirements related to potentially dangerous tasks
- Requirements for reviewing information and making improvements
- Requirements related to known weaknesses and problem areas
- Requirements that come directly from customers
Although there are thousands of requirements organizations must satisfy, and they come from a range of sources, auditors only care about the auditable requirements that are within the scope, or boundary, of the audit. For instance, if I’m doing an ISO 9001 audit, I’m probably not going to pay much attention to regulatory requirements about used oil disposal. This requirement is related to environmental regulatory compliance and is not within the scope of most ISO 9001 audits.
The other caveat related to auditable requirements is that they must be officially approved by company management. If we’re talking about a company procedure or document, the approval usually comes from some sort of signature or other sign-off. If we’re talking about a standard such as ISO 9001, the approval comes from top management making an official declaration to pursue the standard. Usually the only requirements that will be made available to you before and during the audit are ones that are officially approved. If you stick to these, you will be fine.
Let’s talk about a number of different requirements that are likely to be used during an audit.
Company procedures and instructions
This group constitutes the largest category of requirements during most internal audits. They are documents written by the company itself to address its specific needs. They can be long or short, graphic or text, simple or complex. There is no right or wrong way to write a procedure, and they are certainly not written for the convenience of auditors.
The auditor will request to review company procedures prior to the audit and will attempt to understand them during his or her preparation time. Preparation time could take 30 minutes or eight hours depending on the number of requirements within the audit’s scope and the auditor’s familiarity with the organization.
Organizations use a wide range of names to refer to their documents. Some have highly formal categories of documents that relate to different degrees of detail. For instance, you may encounter a company that has a category called “operating procedures” and a category called “work instruction.” Historically, an operating procedure addresses a process from a high level, covering broad systematic requirements. For instance, you may find a document called “Calibration Operating Procedure.” This is likely to be a fairly high-level document that talks about the entire calibration process and how it is managed. On the other hand, a work instruction is usually a much lower level document that addresses tasks from a step-by-step level. For example, you might encounter a “Caliper Calibration” work instruction. This would describe step-by-step how to calibrate a caliper. Please note that organizations are not required to have different levels of documents like this. I only mention it so that you can be aware of different approaches that companies may take.
Besides procedures and work instructions, there are many different “procedural” documents that an organization may decide to write. These include:
- Flow charts
- Checklists
- Process descriptions
- Set-up sheets
- Process standards
- Test methods
- Standard operating procedures
The list goes on and on. As an auditor, you will never be 100 percent familiar with all documents written by the organization. How could you? Your challenge is to do a reasonable amount of preparation so that you can ask meaningful questions during the audit. Some company documents will only rear their heads during the audit, especially in the case of very low-level instructions. You will use these as you see fit, based on their apparent importance and risk.
Company policies and objectives
Policies are the highest level documents written by the organization. They rarely include procedural content (i.e., “how to”), but most often address overall company direction, philosophy, and goals. Since policies are so high level, they can be very difficult to audit against. Nonetheless, they must be included. Within the ISO 9001 framework, the quality policy is specifically required.
Auditors should examine the commitments stated in policies and ask how high-level commitments are deployed in all levels of the organization. A hollow commitment that nobody understands is worthless. If you read a policy that includes a statement such as, “We are committed to becoming the leader in technological innovation,” top management should be asked how they are working toward that commitment. Look for specific evidence that lower level employees also understand the commitments and can explain how they support the policies.
Objectives are also high-level documents, although they have a sharper edge than most policies. Objectives state metrics and measurable goals that the company is pursuing. Here are some examples:
- Reduce warranty repairs to less than one percent of sales.
- Deliver 97 percent on-time shipping.
- Perform all service calls performed within two hours of scheduled appointment.
- Reduce scrap by five percent.
- Increase sales by 10 percent.
As an auditor, you’re concerned about what sort of plans are behind the objectives. Have resources been determined, actions planed, responsibilities identified, and timelines established? Ultimately, is the organization making progress against each objective? If not, what is being done about it? Inherent in the presence of objectives is the obligation to work toward them in a systematic way.
Standard requirements
Most management systems that use internal auditing are based on a standard of some sort. The most common standard is ISO 9001, but there are many others. These standards are usually written to apply to a wide variety of organizations, which tends to make them a little difficult to interpret. For this reason, standard requirements are sometimes de-emphasized during internal audits. This is a mistake. If an organization has implemented a management system standard, then internal audits definitely need to include the applicable standard’s requirements.
Requirements in standards such as ISO 9001 are written as “shall” statements. The “shalls” are sometimes very specific and sometimes quite vague. Understanding the practical interpretations of a standard represents a specific training need for most internal auditors. When auditors apply standard requirements, they often find that they must explain the requirement to the auditee.
Sometimes auditors find that there are two nearly identical requirements: one from the applicable standard and one from a company document. When this happens, the requirement that should be used is the lowest-level requirement. In other words, the requirement written by the organization. Why would an auditor want to use the company requirement instead of a standard requirement? Because it has more specific relevance to the organization. They felt strongly enough about that particular topic to address it within their own procedures, so any issues raised by the auditor on that topic will presumably grab their attention.
Records
Records aren’t often thought of as sources of requirements during an audit. After all, records are historical and describe what happened in the past. Sometimes this past includes decisions that have been made and must be acted upon. Good auditors will identify these commitments and verify that they have taken place. Examples of records that are often used during audits include:
- Purchase Orders: including the requirements that suppliers must meet. This also obligates the organization to verify that the supplier did what they’re supposed to do. An example is the requirement, “Shipment must include certified test results.” The auditor should check to see that certified test results were indeed sent with the shipment and that the test results meet all requirements.
- Management Review Records: These meetings are led by top management and nearly always include action items that must be implemented. Auditors should scan these records for action items and seek evidence that they were carried out.
- Corrective Action Records: Corrective actions are the formal problem-solving events for organizations. They will always include improvement actions that must be implemented. Auditors must verify that improvements have fully implemented and checked for effectiveness.
- Sales Orders and Contracts: Sales records always include product/service requirements that the organization must fulfill. These could include delivery dates, performance requirements, dimensional specifications, or any number of other variables. Auditors should confirm that employees know about these requirements and that the organization is meeting all of them.
Auditing against opinions
Everybody has opinions. Sometimes these opinions make sense. But opinions do not constitute requirements. An auditor’s opinion is never used as a requirement. The irony of opinions is that the more auditing you do, the more likely you are to develop strong opinions. Auditors must always guard against using their opinions during audits. Opinions are part of a whole family of entities that have no place in most audits, including:
- Opinions
- Best practices
- Neat ideas
- World-class methods
- “What we used to do at my old company”
If you see something during the audit that seems like the wrong way to do things, your job is to find a requirement that the organization committed to. Your opinion of what’s right or wrong is not sufficient.
Most audit systems have a category of finding that falls short of a nonconformity. These go by various names, including observation, concern, recommendation, and opportunity for improvement. This type of finding does not generally require the organization to take corrective action. As such, the findings don’t require a solid requirement and can be based on the opinion of an auditor. But nonconformities are never based on auditor opinions.
Traceable requirements
All requirements used during the audit must be traceable. In other words, you can say where each requirement has come from. Requirements without traceability have no credibility or context. When identifying a requirement, the following elements are defined:
- Document name (where the requirement is taken from)
- Document number
- Revision level
- Section number/title
- The exact requirement, taken word-for-word from the source
Here is an example of a well-written requirement, along with the evidence that makes it a nonconformity:
Requirement: The Finishing Procedure (SOP #QOP-32, revision 3) states in section 6.5 that employees must wear white gloves when handling finished product.
Evidence: The auditor observed two employees in the warehouse handling a pallet of finished product (part #443) without white gloves.
View the companion article “Objective Evidence: An Auditor’s Secret Weapon” here.
About the author
Craig Cochran is the North Metro Regional Manager with Georgia Tech’s Economic Development Institute. He has assisted more than 5,000 companies since 1999 in QMS implementation, problem solving, auditing, and performance improvement. Cochran is a Certified Quality Manager, Certified Quality Engineer, and Certified Quality Auditor through the American Society for Quality. He is certified as a QMS Lead Auditor by Certus Professional Certification.
He is the author of numerous books, including ISO 9001:2015 in Plain English and Internal Auditing in Plain English, from which this article was excerpted.
Copyright 2016 by Craig Cochran. All rights reserved.
