8 INTERNAL QUALITY CONTROL OF DATA 8.1 Introduction 8.2 Rounding and Significant figures ‘Control charts 8.4 Preparation of a Control Sample 8.5 Complaints le-shooting 8.1 Introduction In the preceding chapters basic elements of quality assurance were discussed. All activities associated with these aspects have one aim: the production of reliable data with a minimum of errors. The present discussion is concerned with activities to verify that a laboratory produces such reliable data consistently. To this end an appropriate programme of quality control (QC) must be implemented. Quality control is the term used to describe the practical steps undertaken to ensure that errors in the analytical data are of a magnitude appropriate for the use to which the data will be put. This means that the (unavoidable) errors made are quantified to enable a decision whether they are of an acceptable magnitude and that unacceptable errors are discovered so that corrective action can be taken and erroneous data are not released. In short, quality control must detect both random and systematic errors, In principle, quality control for analytical performance consists of two complementary activities: internal QC and external QC. The internal QC involves the in-house procedures for continuous monitoring of operations and systematic day-to-day checking of the produced data to decide whether these are reliable enough to be released. The procedures primarily monitor the bias of data with the help of control samples and the precision by means of duplicate analyses of test samples and/or of control samples. These activities take place at batch level (second-line control). The external QC involves reference help from other laboratories and participation in national and/or international interlaboratory sample and data exchange programmes (proficiency testing; third-line contro). The present chapter focuses mainly on the internal QC as this has to be organised by the laboratory itself. External QC, just as indispensable as the internal QC, is dealt with in Chapter 9. 8.2 Rounding and Significant figures 8.2.1 Rounding 8.22 Significant figures At this point, before entering into actual treatment of data, it might be useful to enter into the data themselves as they are treated and reported. Analytical data, either direct readings (e.g. pH) of results of one or more calculation steps associated with most analytical methods, are often reported with several numbers after the decimal point. In many cases this suggests a higher significance than is warranted by the combination of procedure and test materials. Since clear rules for rounding and for determining the number of significant decimals are available these will be given here.8.2.1 Rounding To allow a better overview and interpretation, to conserve paper (more columns per page), and to simplify subsequent calculations, figures should be rounded up or down leaving out insignificant numbers. ~ To produce minimal bias, by convention rounding is done as follows: - If the last number is 4 or less, retain the preceding number; - if itis 6 or more, increase the preceding number by 1; - if the last number is 5, the preceding number is made even. Examples. pH = 5.72 rounds to 5.7 pH = 5.76 rounds to 5.8 pH = 5.75 rounds to 5.8 pH = 5.85 rounds to 5.8 When calculations and statistics have to be performed, rounding must be done at the end. Remark: Traditionally, and by most computer calculation programs, when the last number is 5, the preceding number is raised by 1. There is no objection to this practice as long as it causes no disturbing bias, e.g. in surveys of attributes or effects. 8.2.2 Significant figures 8.2.2.1 Rounding of test results 2.2 Rounding of means and standard deviations 8.2.2.1 Rounding of test results The significance of the numbers of results is a function of the precision of the analytical method. The most practical figures for precision are obtained from the own validation of the procedure whereby the -within-laboratory standard deviation s, (between-batch precision) for control samples is the most realistic parameter for routine procedures (see 7.5.2). For non-routine studies, the s, (within-batch precision) might have to be determined To determine which number is stil significant, the following rule is applied: Calculate the upper boundary by of the rounding interval a using the standard deviation s of the results (n> 10) b= %s (8.1) Then choose a equal to the largest decimal unit (100; 10; 1; 0.1; 0.01; etc.) which does not exceed the calculated by ‘After having done this for each type of analysis at different concentration or intensity levels it will become apparent what the last significant figure or decimal is which may be reported. This exercise has to be repeated regularly but is certainly indicated when a new technique is introduced or when analyses are performed in a nonroutine way or on non-routine test materials. Example Table 8-1. A series of repeated CEC deter each in a different batch. ations (in cmol,/kg) on a control sample,[Datal[Rounded| (6.55 6.6 7.01|_7.0 725, 72 783, 78 [6.95] 7.0 7-16)[_ 72 7.83)|_ 78 7.05)|_7.0 6.83) 68 763, 7.6 ‘The standard deviation of this set of (unrounded) data is: $=0.4298 hence: by = 0.2149 and: a=0.1 Therefore, these data should be reported with a maximum of one decimal. 8.2.2.2 Rounding of means and standard deviations When values for means, standard deviations, and relative standard deviation (RSD and CV) are to be rounded, b is calculated in a different way: for p 7), Do not use too small samples for the analysis as this will adversely affect the representaliveness resulting in an unnecessary high standard deviation. Note. A sample may prove to be homogeneous for one attribute but not for another. Therefore, fundamentally, homogeneity of control samples should be tested with an analysis for each attribute for which the control sample is used. This is done for certified reference samples but is often considered too cumbersome for laboratory control samples. On the other hand, such an effort would have the additional advantage that useful information about the procedure and laboratory performance is obtained (repeatability), Also, such values can be used as initial values of control charts. Check on the Mean (sample bias) This is a check to establish if all samples belong to the same population. The means of the duplicates are calculated and treated as single values (x)) for the samples 1 to n. Then, using Equations (6.1) and (6.2), calculate ~x and s of the data set consisting of the means of duplicates (include all data, i.e. do not exclude outliers). |. 8-6. Scheme for the preparation and homogen test of control samples. homogenized bulk sample laboratory 1 2 3 4 Sj--]n control samples nN A A A A AN AB AB AB AB AB AB Gbieate The rules for interpretation may vary from one laboratory to another and from one attribute to another. In general, values beyond + 2s from the mean are considered outliers and rejected. The sample container concerned may be discarded or analyzed again after which the result may well fall within x + 2s and be accepted or, otherwise, the subsample may now definitely be discarded. Check on the Range (sample homogeneity) This is a check to establish if all samples are homogeneous. The differences R between duplicates of each pair are calculated (include all data, i.e, do not exclude outliers). Then calculate R and sp of the data set using Equations (8.5) and (8.6) respectively. The interpretation is identical to that for the Check on the Mean as given in the previous paragraph. Thus, a laboratory control sample container may have to be discarded on two grounds: 1. because it does not sufficiently represent the level of the attribute in the control sample and2, because it is intemally too heterogeneous. The preparation of a control sample including a test for homogeneity should be laid down ina SOP. Example In Table 8-3 an example is given of a check for homogeneity of a soil control sample of 5 kg which was spiit into ten equal laboratory control samples of which the loss-on-ignition was determined in duplicate. The Joss-on-ignition can be determined as follows: 1. Weigh approx. 5 g sample into a tared 30 mL porcelain crucible and dry overnight at 105°C. 2. Transfer crucible to desiccator to cool; then weigh crucible (accuracy 0.001 g). 3. Place crucibles in furnace and heat at 900°C for 4 hours. 4, Allow furnace to cool to about 100°C, transfer crucible to desiccator to cool, then weigh crucible with residue (accuracy 0.001 g) Now, the weight loss between 110 and 900°C can be calculated and expressed in mass % or in g/kg (weight basis: material dried at 105 °C). Table 8-3, Results (in mass/mass %) of duplicate Loss-on-Ignition determinations (A and B) on representative subsamples of ten 500 g laboratory samples of a soil control sample, [Sample] A |B [Meanas] je] (9.10/8.42| 8.760 ||0.68| (9.65 /6.66| 9.155 [0.99] (9.63/9.18| 9.405 |[0.45) (8.65|8.89| 8.70 |[0.24) (8.77 |9.19| 8.950 |[0.48| (9.14|8.93[ 9.040 [0.22] (a.71|8.97| 8.840 ||0.26| (8.59/8.78| 8.685 [0.19] (8.86 /9.12| 8.990 |[0.26) (9.04|8.75| 8.895 |[0.29| Mean: 8.949 0406 8 0.214" Sp: 0.334" (using Eq. 6.2; “ using Eq. 8.6) Tolerance range for mean of duplicates (“x + 2s): 8,949 + 2 * 0,214 = 8,52-9.38% Tolerance range for difference R between duplicates: jo4ne 2 x0334|=0-LoM% In this example it appears that only the mean result of sample no. 3 (= 9.405%) falls outside the permissible range. However, since this is only marginally so (less than 0.3% relative) we may still decide to accept the sample without repeating the analysis.The measure R for internal homogeneity falls for all samples within the permissible range. (Should an R be found beyond the range we may opt for repeating the duplicate analysis before deciding to discard that sample.) 8.5 Complaints Errors that escaped detection by the laboratory may be detected or suspected by the ‘customer. Although this particular type of quality control may not be popular, it should in no case be ignored and can sometimes even be useful. For the dealing with complaints a protocol must be made with an accompanying Registration Form with at least the following items: name client, and date the complaint was received - work order number - description of complaint - name of person who received the complaint (usually the head of laboratory) - person charged with investigation - result of investigation - name of person(s) who dealt with the complaint - an evaluation and possible action - date when report was sent to client Arecord of complaints should be kept, the documents involved may be kept in the work order file. The trailing of events (audit trailing) may sometimes not be easy and particularly in such cases the proper registration of all laboratory procedures involved will show to be of great value. Note. Registration of procedures formally also applies to work that has been put out to contract to other laboratories. When work is put out, the quality standards of the subcontractor should be (demonstrably) satisfactory since the final responsibility towards the client lies with the laboratory that put out the work. If the credibility needs to be verified this is usually done by inserting duplicate and blind samples. 8.6 Trouble-shooting Whenever the quality control detects an error, corrective measures must be taken. As mentioned earlier, the error may be readily recognized as a simple calculation or typing error (decimal point!) which can easily be corrected. If this is not the case, then a systematic investigation must take place. This includes the checking of sample identification, standards, chemicals, pipettes, dispensers, glassware, calibration procedure, and equipment. Standards may be old or wrongly prepared, adjustable pipettes may indicate a wrong volume, glassware may not be cleaned properly, equipment may be dirty (e.g. clogged burner in AAS), or faulty. Particularly electrodes can be a source of error: they may be dirty and their life-time must be observed closely. ApH-electrode may seemingly respond well to calibration buffer solutions but still be faulty. Clearly, every analytical procedure and instrument has its own characteristic weakness, by experience these become known and it is useful to make a list of such relevant check points for each procedure and adhere this to the corresponding SOP or maintenance logbook if it concerns an instrument. Update this list when a new flaw is discovered Trouble-shooting is further discussed in Section 9.4. 8.7 LIMS 8.7.1 Introduction 8.7.2 What is a LIMS? 8.7.3 How to select a LIMS.8.7.1 Introduction The various activities in a laboratory produce a large number of data streams which have to be recorded and processed. Some of the main streams are: - Sample registration - Desired analytical programme = Work planning and progress monitoring - Calibration - Raw data - Data processing - Data quality control - Reporting = Invoicing - Archiving Each of these aspects requires its own typical paperwork most of which is done with the help of computers. As discussed in previous chapters, it is the responsibility of the laboratory manager to keep track of all aspects and tie them up for proper functioning of the laboratory as a whole. To assist him in this task, the manager will have to develop a working system of records and journals. In laboratories of any appreciable size, but even with more than two analysts, this can be a tedious and error-sensitive job. Consequently, from about 1980, computer programs appeared on the market that could take over much of this work. Subsequently, the capability of Laboratory Information Management Systems (LIMS) has been further developed and their price has increased likewise The main benefit of a LIMS is a drastic reduction of the paperwork and improved data recording, leading to higher efficiency and increased quality of reported analytical results, Thus, a LIMS can be a very important tool in Quality Management. 8.7.2 What is a LIMS? The essential element of a LIMS is a relational database in which laboratory data are logically organized for rapid storage and retrieval. In principle, a LIMS plans, guides and records the passage of a sample through the laboratory, from its registration, through the programme of analyses, the validation of data (acceptance or rejection), before the presentation and/or filing of the analytical results, Hardware Originally, LIMSes were installed on mainframe and minicomputers in combination with terminals. However, with the advent of stronger PCs, programs were developed that ‘could run on a single PC (single-user system) or on several PCs with a central one acting as server (network, multi-user system). The more expensive systems allow advanced automation of a laboratory by direct coupling of analytical instruments to the system, Printers are essential parts of the system for label and bar code printing as well as for graphs and reports. Software The LIMS software consists of two elements: the routines for the functional parts, and the database. For the latter usually a standard database program is used (e.g. dBase, Oracle,) which can also be done for certain functional parts such as production of graphs and report generation The database is subdivided into a static and a dynamic part. The static part comprises the elements that change only little with time such as the definition of analytical methods, whereas the dynamic part relate to clients, samples, planning, and results. Function features -Anumber of common main features of a LIMS are the following:- Registration of samples and assigned jobs with unique numbers and automatic label production. - Production of work lists for daily and long-term planning. - Allows rapid insight in status of work (pending jobs, back-log). - Informs about laboratory productivity (per analysis, whole laboratory). - Production of control charts and signalling of violation of control rules (results beyond Action Limit, etc.) - Flagging results beyond preset specifications. - Generates reports and invoices. ~ Archiving facility. Allows audit trailing (search for data, errors, etc.) Data collection and subsequent calculations are usually done “outside” the LIMS. Either with a pocket calculator but more commonly on a PC with a standard type spreadsheet program (such as Lotus 123) or with one supplied with the analytical instrument. The data are then transferred manually or, preferably, by wire or diskette to the LIMS. The larger LIM systems usually have an internal module for this processing. ‘Amajor problem with the application of a LIMS is the installation and the involved ‘customizing to the specific needs of a laboratory, One of the first asked questions (after asking for the price) is: ‘can | directly connect my equipment to the LIMS?’. Invariably the answer of the vendor is positive but the problems involved are usually concealed or unjustly trivialized. It is not uncommon that installations take more than a year before the systems are operational (not to speak of complete failures), and sometimes the performance falls short of the expectations because the operational complexity was underestimated. Mentioning these problems is certainly not meant to discourage the purchase of a LIMS. On the contrary, the use of a LIMS in general can be very rewarding. Itis rather intended as a warning that the choice for a system must be very carefully considered, 8.7.3 How to select a LIMS When it is considered that a computerized system might improve the management of the laboratory information data flow, a plan for its procurement must be made. The most important activities prior to the introduction of a LIMS are the following: - Set up LIMS project team. Include a senior laboratory technician, the future system manager and someone from the computer department. - Review present procedures and workload. - Consider if a LIMS can be useful Define what the system must do and may cost (make cost/benefit assessment). The cost/benefit assessment is not always straightforward as certain benefits are difficult to ‘assess or to express in money (e.g. improved data quality; changing work attitude). A\so, a LIMS may be needed as a training facility for students. When a decision is made that a LIMS project is viable, the team must define the requirements and consider the two ways to acquire a LIMS: either by in-house building a system or by purchasing one. Many in-house systems are not premeditated but result from a gradual build-up of small programs written for specific laboratory tasks such as the preparation of work lists or data reports. The advantage is that these programs are fully customized. Thedisadvantage is that, lacking an initial master plan, they are often not coupled or integrated into an overall system which then takes extra effort. Yet, many laboratories employ such "systems". If a system has to be built from scratch then the general rule is that if a suitable commercial package can be found, it is not economical to build a system as it is both a complicated and time-consuming process. The purchase of a commercial LIMS should be a well structured exercise, particularly if a large and expensive system is considered. Depending on the capabilities, prices for commercial systems range from roughly USD 25,000 to 100,000 or even higher. The next steps to be taken are: - Identify LIMS vendors. - Compare requirements with available systems. - Identify suitable systems and make shortlist of vendors. - Ask vendors for demonstration and discuss requirements, possible customization, installation problems, training, and after-sales support. - If possible, contact user(s) of candidate-systems. After comparing the systems on the shortlist the choice can be made. By way of precaution, it may be wise to start with a “pilot” LIMS, a relatively cheap single-user system in part of the laboratory in order to gain experience and make a more considered decision for a larger system later. Itis essential that all laboratory staff are involved and informed right from the start as a LIMS may be considered meddlesome (‘big brother is watching me’) possibly arousing a negative attitude. Like Quality Management, the success of a LIMS depends to a large extent on the acceptance by the technical staff. Remark: Useful information can be obtained from discussions by an active LIMS working group of several hundreds of members on Internet, To subscribe to the (free) mailing list send an E-mail message to: listproc@govonea?.gov.on.ca stating after Subject: subscribe ims .. (fillin your name). Another site one may try is: http://www.limsource.com. SOILIMS lAn example of a low-budget and simple stand-alone LIMS specially built for small to medium- Isized soil, plant and water laboratories is SOILIMS. It is a user-friendly system which is easily ‘stalled and learned (the manual contains a Tutor) and can be used immediately after installation. Although the system has about 100 analyses in the standard configuration, it can lbe farther customized (and re-customized later) by the supplier. A unique feature is that more Ithan a dozen different cross-checks can automatically be performed in order to screen soil data, for internal inconsistencies: when "anomalies" occur, the data concerned are flagged for closer linspection before they are released (anomalities do not necessarily imply errors in all cases). |An attractive feature is its price which is comparable to that of a bench-top pH meter. The main lteatures are the following while the system’s main menu is given below”. |. Unambiguous registration by automatic assignment of unique work order and laboratory lsample numbers. | Possibility of priority assignments by deadline definition. | Flexibility to alter work order requests and deadlines. |. Time-saving routine for sample label production. | Protection of data against non-authorized users. | Backlog reporting, | Detailed information regarding the status of pending work orders.|. Production of work lists provides the manager with complete and accurate information for fast \decision making. | Allows for many control samples. |: Manual or automatic data input (direct ASCII file reading). | Second-line control by automatic verification of control sample results in Control Charts, | Unique capabilities for cross-checking data (‘artifical intelligence"). | Increased efficiency by easy production of reports and invoices. |. Data expor facilities to LOTUS 123 or text editors. | Easy-to-use automatic archival procedures. | Audit rail capabilities for specified samples, clients, work orders, or laboratory personnel. | Stand-alone and single-user network version. | Option for plant and water analysis included | Millennium proof, | For more information contact: ISRIC, P.O, Box 353, 6700 AJ Wageningen, the Netherlands. E- Imail: aboratory@isric.nt [Minimum required hardware: IBM PC (or compatible) 386 SX with 4Mb RAM. [Figure SOPs Model: Mean Chart Model: Range Chart Model: Combined Mean Chart and Range Chart Model: Mean Chart Model: Mean Chart Model: Range Chart Model: Range Chart Model: Com ied Mean Chart and Range Chart Me mbined Mean Chart and Range Chart El Ey