How Many 'Bad Apples' Are Too Many?
There are many contenders for the event that began the quality movement and in reality there was no single event, but a progression of events that still continue. However, one catalyst that had a major impact was the crisis in World War One concerning the quality of shells. At this time, trench warfare had opposing armies so close to each other that it meant artillery fire had to be more accurate than ever before. Problems existed if a shell fell short or travelled too far. There was an urgent need for shells to be highly consistent in their performance.
Not surprisingly, the military pushed for quality in manufacture and demanded standardised products. In particular, they needed a standard method to determine whether or not to accept a shipment of shells, while not testing every shell, given that the final test was to fire it. This led to the concept of testing just a few shells from every batch and using the test results to decide whether to have confidence in the whole batch. Rather than take risks with a batch they did not trust, the military preferred to reject the entire batch.
Time has refined these concepts, using ideas from probability and statistics, to what is now called Acceptance Testing. Today there are International Standards for acceptance testing, such as ISO 2859. These Standards cover many contexts, but they have their origins in Military Standards. In fact the sampling plans in ISO 2859 are copied without change from Military Specification (MIL-SPEC) 105E, developed by the United States Military. These Standards form the basis of supplier contracts, entitling the customer to reject shipments when many defects are found in a sample.
Data Analysis Australia has worked in the area of Acceptance Sampling for many years. In our experience, while the Standards embody many good ideas, they are not flexible enough for all circumstances. Some clients need a level of quality that the Standards do not directly provide, their batches are not as simple as the Standard assumes or their aims are not exactly what the Standard writers had in mind. The client might have a need for several measurable outcomes instead of a simple accept/reject decision. In these cases it is important to be sure that the sampling methodology and the decision rules are optimised to achieve the desired outcomes at a reasonable cost.
Adapting the concepts in the Standards to new situations requires a full understanding of the statistical theories behind the Standard. Only by this understanding can the intent of the Standard be implemented in a new context. This is where Data Analysis Australia plays a critical role for our clients. Using both our expertise in statistical sampling and our experience in understanding our clients' problems, we are able to develop customised solutions.
An example is a recent project for the Department of Defence, where potential pricing errors recorded in the inventory system raised audit concerns. With an inventory consisting of about 500,000 different types of items and many millions of transactions each year, it was not feasible to check the pricing on each item. With the dual aims of i) identifying areas that needed to improve their pricing performance and ii) enabling an overall correction to the accounts, Data Analysis Australia treated the purchase transactions from each area as a batch. Using Acceptance Sampling principles, poorly performing areas were identified as those that had a batch rejected. At the same time, the sample was optimised to adjust the accounts, fulfilling the second aim.
In a completely different context, Data Analysis Australia assessed the use of the Australian Standard for Acceptance Sampling to measure the quality of a large scale scanning exercise. Hundreds of thousands of documents and files had to be scanned and managed electronically, while maintaining a high degree of confidence that the electronic versions exactly matched the paper files. In this case, overall batches were defined by boxes, but within each box, there was a series of 'nested' batches - testing items within pages, within documents, within folders, within boxes. For a box to be 'accepted', all of the test items had to be passed and any test items that failed in a box had to be rescanned and retested. Our recommendations incorporated random selection of each of these nested batches in an efficient manner, as well as what to do when some of the test items in a box failed.
For further information, please Contact Us.
September 2010