Statistics & graphics for the laboratory 54 Metrological basis of IQC Introduction The error concept chosen here The total error concept Total error –

Slide 1 Statistics & graphics for the laboratory 54 Metrological basis of IQC Introduction The error concept chosen here The total error concept Total error – calculations Instability of the analytical process Instability and analytical process specifications Instability – how much can be tolerated? Analytical process specifications - TEa IQC and TEa The error model for IQC Basic formula Critical errors Graphical presentation of critical errors Calculation of critical errors Special topic: The TEa problematic Control rules based on TEa Automatic selection of rules based on TEa: The Validator OPSpecs®-Charts Critical error graphs Selection of a control rule based on TEa with the Validator: an example Other selection tools: the TEa/CV a,tot ratio; the IQC decision tool Summary and Conclusion Slide 2 Statistics & graphics for the laboratory 55 Metrological basis of IQC Introduction FOREWORD Currently, there is confusion about metrological terms. While the term error is used in the metrological “bible” (VIM [1]), it is deprecated in one of the recent ISO concepts (GUM [2]). This concept, even, recommends to avoid the distinction between random and systematic effects (= uncertainty concept: GUM). Moreover, ISO uses the concept of accuracy [3]: Accuracy = Combined trueness & precision Inaccuracy = Combined ([untrueness]: note: this term is not defined by ISO) and imprecision. Note: Some traditions link accuracy to systematic error. This book uses the [total] error concept (Adapted from: Stöckl D. Scand J Clin Lab Invest 1996;56:193-7): Total-, systematic-, and random error [In]Accuracy: combined [“un“]trueness and [im]precision Experimental estimates: Total error, bias, SD Note: all experimental estimates have a statistical uncertainty! References [1] ISO VIM. Vocabulaire international des terms fondamentaux et généraux de métrologie. [2] ISO GUM. Guide to the expression of uncertainty in measurement. [3] ISO 5725-1. Accuracy (trueness and precision) of measurement methods and results. Classical metrological error concept: distinction of random and systematic error. Inaccuracy, “untrueness“ and imprecision: pendents for total-, systematic-, and random error. Note: All experimental values (bias, s) have a certain statistical level of probability («±uncertainty»). Slide 3 Statistics & graphics for the laboratory 56 Metrological basis of IQC The total error (TE) concept The term "total error" (TE) (#) is applied to measurement results that are influenced by random (RE) and systematic (SE) components of error. Calculation of TE for single measurements: TE = SE + z RE, or TE = SE + z s –(z is usually set to 1.96 or 2.58, encompassing 95% or 99% [two-tailed] of a gaussian distributed population) Calculation of TE for multiple measurements: TE = SE + t n-1 [s/  n] –s = the standard deviation, n = number of measurements, t n-1 = student's t- value for (n-1) degrees of freedom (e.g., at the 95 % confidence level) Note: Random error can be reduced by increasing the number of measurements, systematic error cannot. Instability of the analytical processes -Due to occurrence of systematic error (1) -Due to increased random error (2) References #Random, systematic, and total error of measurement [adapted from: Westgard JO, Barry PL. Cost-effective quality control. 4th printing. Washington: AACC Press, 1995] Slide 4 > TE Stable, "very loose" IQC-rules can be selected. Prerequisite for IQC TE stable < TEa There must be some room left for the IQC-procedure Note: IQC cannot transform an intrinsically poor method into a good method #References Strategies to set global analytical quality specifications in laboratory medicine. Consensus Statement, Stockholm 1999. Scand J Clin Lab Invest 1999;59:585 Stöckl D, Baadenhuijsen H, Fraser CG, Libeer J-C, Hyltoft Petersen P, Ricós C. Desirable routine analytical goals for quantities assayed in serum. Eur J Clin Chem Clin Biochem 1995;33:157-69. www.westgard.com Metrological basis of IQC Instability and analytical process specifications How much instability? TE IQC, max < TEa  work within specifications TE IQC,max : Maximum total error guaranteed by the IQC-procedure (95% of results should be within ±TEa)"> Statistics & graphics for the laboratory 57 Instability – how much can be tolerated? The answer is:  Poor analytical quality must not invalidate the medical decision! The conclusion is: We have to know the medical requirements for analytical quality («analytical process specifications»)  Allowable total error (TEa). Analytical process specifications - TEa Concepts and numerical examples can be found in the literature below (#). Note: Optimally, TEa should be medically justified! IQC and TEa Observation The statistically defined "stop-limit" of IQC (TE IQC ; e.g., 3 s) may be much lower than medically relevant (e.g., triacylglycerides). The idea Let's stop the process at a medically relevant (or otherwise defined) allowable total error: TEa (TEa: also called "quality specification"). Advantages If TEa >> TE Stable, "very loose" IQC-rules can be selected. Prerequisite for IQC TE stable < TEa There must be some room left for the IQC-procedure Note: IQC cannot transform an intrinsically poor method into a good method #References Strategies to set global analytical quality specifications in laboratory medicine. Consensus Statement, Stockholm 1999. Scand J Clin Lab Invest 1999;59:585 Stöckl D, Baadenhuijsen H, Fraser CG, Libeer J-C, Hyltoft Petersen P, Ricós C. Desirable routine analytical goals for quantities assayed in serum. Eur J Clin Chem Clin Biochem 1995;33:157-69. www.westgard.com Metrological basis of IQC Instability and analytical process specifications How much instability? TE IQC, max < TEa  work within specifications TE IQC,max : Maximum total error guaranteed by the IQC-procedure (95% of results should be within ±TEa) Slide 5 5% of the results exceed TEa. Graphical presentation of critical errors References # Westgard JO, Barry PL. Cost-effective quality control. 4th printing. Washington: AACC Press, 1995] Introduction of an SE that places 5% of results outside TEa:  SE crit = 2 Increase of an RE that places 5% of results outside TEa:  RE crit = 2 (RE i = intrinsic or stable imprecision)"> Statistics & graphics for the laboratory 58 Metrological basis of IQC The error model for IQC Basic formula (#) TE = Bias + [  SE cont s meas ] + z [  RE cont s meas ] Bias: method bias, usually assumed to be zero  SE cont : change in systematic error that can be detected by the quality control procedure s meas : measured, intrinsic standard deviation of the test  RE cont : change in random error that can be detected by the quality control procedure z: statistical multiplier related to the portion of a distribution exceeding a quality requirement ("defect rate") Note: mostly, a z-value of 1.65 is selected to obtain a 5% defect rate (one-tailed). When pure random error is addressed, a z-value of 1.96 is selected to obtain a 5% defect rate (two-tailed). Critical random and systematic error definition: The amount of error that places 5% of results outside TEa. The role of IQC is to detect process deterioration before >5% of the results exceed TEa. Graphical presentation of critical errors References # Westgard JO, Barry PL. Cost-effective quality control. 4th printing. Washington: AACC Press, 1995] Introduction of an SE that places 5% of results outside TEa:  SE crit = 2 Increase of an RE that places 5% of results outside TEa:  RE crit = 2 (RE i = intrinsic or stable imprecision) Slide 6 Statistics & graphics for the laboratory 59 Metrological basis of IQC Calculation of critical errors Assumption of imprecision and bias Critical random error (  REc) and critical systematic error (  SEc) can be calculated that cause measurements to exceed TEa in 5% of the cases (= defect rate) due to either increased imprecision or increased systematic error. Formulae for a 5% defect rate: Case with  SEc = 0  REc = (TEa - bias)/1.96 s meas ; Case with  REc = 1  SEc = [(TEa - bias)/s meas ] - 1.65 Assumption of zero bias For the purpose of IQC, the bias of the method often is assumed to be zero. This is justified if a laboratory establishes its own IQC target values or if it can verify the system-specific target values of commercial IQC materials. Formulae for  REc and  SEc with bias = 0  REc = TEa/1.96 s meas ;  SEc = (TEa/s meas ) - 1.65 Note: For simplification, the QC-Validator® program applies the default factor 2 (for RE and SE) for the stable process (elder versions: 1.65!). The factor may be changed by the user. Checklist – Metrology and rule selection Metrology, knowledge of The total error concept (TE, SE, RE) Concept of maximum allowable total error (TEa) (process specifications) Critical error Rule selection Statistical basis: A rule is chosen based on P fr and P ed. SD-limits are taken from stable performance. TEa basis: From a specification for TEa, critical error values can be calculated. From the critical error values, adequate IQC rules can be selected, naturally, on statistical basis. Selection tools: power functions, OPSpecs, the TEa/CV a,tot ratio; the IQC decision tool Slide 7 Statistics & graphics for the laboratory 60 Special topic: the TEa problematic Introduction Specifications for TEa, the concepts  Clinical  Biological –Notes on IQC rules based on biological TEa –IQC rules from biological TEa and reality  Expert Summary Checklist – TEa values Slide 8 Statistics & graphics for the laboratory 61 Special topic: the TEa problematic Introduction BEWARE There are a lot of misunderstandings when selecting a TEa for IQC purposes.  We look into TEa in more detail! Questions What is a responsible choice for TEa? Is there a consensus about TEa?  Investigate current concepts for establishing values for TEa. Specifications for TEa Current concepts for TEa The Table below lists, in hierarchical order, current concepts for establishing values for TEa. Sources (in hierarchical order#) Clinical concepts (few analytes) Concepts based on biological variation Expert opinion Regulations "State-of-the-art" #Consensus Statement (Stockholm 1999). Scand J Clin Lab Invest 1999;59:585. See also: ISO 15196 Slide 9 Statistics & graphics for the laboratory 62 Special topic: the TEa problematic Specifications for TEa Inspection of proposed numbers for TEa  EXAMPLE: Serum-sodium When we compare the currently proposed numbers, we can conclude that: There is no consensus about TEa! Additionally: TEa may depend on concentration/disease Slide 10 Statistics & graphics for the laboratory 63 Special topic: the TEa problematic TE from biology BUT: Some are too stringent for current technology Biology gives extreme low values for some analytes: TEa sodium: Ricos: 0.9% (http://www.westgard.com/biodatabase1.htm; but note: CV g on that site = CV b )http://www.westgard.com/biodatabase1.htm  Apply a bottom-line# #Stöckl D. Desirable Performance criteria … based on biological analyte variation - hindrances to reaching some and reasons to surpass some. Clin Chem 1993;39:913-4. Bottom-line (for stable process!) CV: 1% Bias: 1,5% TE: 3,2% Consider to surpass some Think about using more stringent goals than derived from biology  the goals are a compromise; account for IQC; some goals are surpassed, by far, by current technique. Stöckl D. Desirable Performance criteria … based on biological analyte variation - hindrances to reaching some and reasons to surpass some. Clin Chem 1993;39:913-4. Tonks: 10% upper limit for TE Slide 11 Statistics & graphics for the laboratory 64 Special topic: the TEa problematic Proposed numbers – How realistic are they? The numbers proposed by clinical and biological concepts, usually, are desirable numbers (= goals), which would allow medical decisions to be made without compromise of analytical quality. However, the laboratory has to work with “real numbers” = “state-of-the-art” quality. Comparison of reality with “goals” from biology The Figure below compares “ state-of-the-art” quality with desired quality: 1/4 of the reference interval. [1] TEa = ¼ of the reference interval [2] State-of-the-art total CVa. *Rules calculated with the above TEa and CV values by the Validator (90% assurance; 2 materials; bias: Na 0.5%, Trigl 2%, others: 1%). Observation Some state-of-the-art CVs are too big to allow calculation of IQC-rules from biology-derived TEa values, for example, for  sodium. Expert TEa values  Sodium: other numbers Koch, Westgard et al. Clin Chem 1990;36:230-3. We have seen that sodium was a problem for IQC with TEa values derived from biology. For others, however, sodium seems to be very easy for IQC (see below). They propose a 3.5 s rule. Note: TEa was chosen by experts and was the US EQA-limit (2.9%). Additional, their “state-of-the-art” CV (0.5%) seems to be quite optimistic! CONCLUSION Don’t cheat yourself by selecting the most liberal TEa values and by underestimating your actual CV. Slide 12 Statistics & graphics for the laboratory 65 More on expert TEa values Below, are shown more examples of IQC rule setting by use of expert TEa values (upper example) and by general expert opinion (lower example). Koch, Westgard, et al. Clin Chem 1990;36:230-33 (see also : Westgard & Stein. Clin Chem 1997;43:400-3). Based on “expert” TEa values, following rules were proposed: 1 3.5s (n = 2) for: Sodium (TEa = 2.9%, CV = 0.5%), potassium, urea nitrogen, creatinine, phosphate, uric acid, cholesterol, total protein, total bilirubin, GGT, ALP, AST, and LD. 1 2.5s (n = 2) for: Chloride, total CO2. Calcium was a problem: Compared to TEa, precision was considered as too poor. Mugan, Carlson, Westgard. J Clin Immunoassay 1994;17:216-22. 1 2.5s (n = 3) for: Prolactin, ß-hCG, CEA, FSH, LH, TSH, ß2-microglobulin.  Experts tend to simple rules; choice is not directly related to medical requirements Rules selected by general opinion Steindel SJ, Tetrault G. Arch Pathol Lab Med 1998;122:401-8. 2.5 – 2.7s rule: generally Tetrault GA. CAP Today 1995 (April):60-1. 3.5 s rule: generally  Experts tend to simple rules; choice is often more related to considerations about P fr than to medical requirements. Sometimes, experts change their minds. Observations  Experts tend to simple rules; choice is not directly related to medical requirements  Choice is often more related to considerations about P fr than to medical requirements. Sometimes, experts change their minds. Special topic: the TEa problematic Slide 13 Statistics & graphics for the laboratory 66 Special topic: the TEa problematic TEa and IQC Summary Different models give different TEa values Additionally: TEa may depend on concentration/disease Many proposals do not account for the needs of IQC Some values that are recommended are beyond the state-of-the-art of routine methods, or give IQC rules with high P fr Experts tend to simple rules; choice is often more related to considerations about P fr than to medical requirements. Sometimes, experts change their minds. Don’t choose the most convenient TEa. The most useful general purpose TEa numbers result from the concept of biological variation. –But, apply bottom-line values Checklist – TEa values Apply TEa values from the following hierarchy 1. Clinical models (e.g., cholesterol; glucose) 2. Biological variation (obtain the database) Bottom-line values No numbers from 1-2: 3. Expert models No numbers from 1-3: 4. Regulation No numbers from 1-4: 5. Better state-of-the-art Note: Critically review the proposed numbers. Slide 14 Statistics & graphics for the laboratory 67 Control rules based on TEa Automatic selection of rules based on TEa: The Validator OPSpecs®-Charts Critical error graphs Selection of a control rule based on TEa with the Validator: an example Other selection tools: the IQC decision tool; the TEa/CV a,tot ratio; Summary and Conclusion Slide 15 Statistics & graphics for the laboratory 68 Control rules based on TEa Selection of control rules based on TEa Tools for IQC rule decision based on TEa The Westgard software tools (OPSpecs® chart) The STT IQC decision tool The ratio of TEa/total-CVa: should be >4 Automatic selection of rules based on TEa: The Validator® Based on the TEa concept and the calculation of critical errors, the Validator software allows an automatic selection of the most appropriate control rule. OPSpecs-charts The Validator makes use of so-called OPSpecs-charts (“Charts of Operational Process Specifications”) to relate TEa and stable method performance (bias and imprecision) with IQC-rule selection. Based on a preselected TEa, these charts allow to select IQC rules for different combinations of bias/imprecision (see Figure). The charts are specific for a specific total error requirement, for example, a chart for a TEa of 5% will differ from one requiring 2%. A selection between QC procedures for systematic error or random error has to be made and the probability level for error detection has to be specified, for example, 90% (80% for RE), 50%, etc (OPSpecs®-Charts for RE not in EZ rules!). Imprecision is plotted on the x-axis and bias is plotted on the y-axis. Stable imprecision and bias are plotted as operating point (Figure: 2%/0%). A selection of IQC-rules is presented. Slide 16 Statistics & graphics for the laboratory 69 Control rules based on TEa Critical error graphs The Validator® Additionally, it shows the power functions of the rules with indication of the critical error (= critical error graphs). Critical error graphs are made for systematic and random error, separately. They show the power functions for the IQC rules that the Validator proposes. Additionally, a box is presented that indicates P fr and P ed of the respective rules. The critical error is indicated by a vertical line. Usually, it should intersect the power functions at P ed values >0.9 (=90% for error detection).  Note: for the RE graph, the critical error line intersects most power functions at values Statistics & graphics for the laboratory 73 As already addressed before, different Validator® versions may use different default k values for the stable process: - Either 1,96: pure RE - Or 1.65: pure SE Usually, both components are present. In practice, there is a gradual move from 1.96 to 1.65 when SE moves from 0 to SE = RE. If SE > RE, the multiplier 1.65 is justified. This has also implications for the IQC controlled process. For this reason, the STT variant was developed. IQC decision tool – the STT variant The variant is based on the OPSpecs® charts. From RE = 0, the lines with k = 1.65 (k = 4) are used up to SE = RE. However, from that point on, a line is drawn to the RE point that corresponds to TE/2 (TE/4.85). Control rules based on TEa Other selection tools Problem of the OPSpecs® charts When RE becomes > SE, one moves from the 1-sided case (k = 1,65; k = 4) to the 2-sided case (k = 2; k = 4,85) Slide 21 Statistics & graphics for the laboratory 74 Control rules based on TEa The ratio of TEa/total-CVa (EXCEL-file) The ratio of TEa/total-CVa allows a quick estimate of test performance. This ratio should be >4. However, the ratio depends strongly on the TEa chosen (see also discussion later: special topic TEa problematic). REMARK: bias is not considered by that tool! The EXCEL-file contains a nearly complete list of TEa and state-of-the-art CVa values for the following analyte groups: Ion selective electrode (ISE) Substrates Enzymes Specific proteins Therapeutic drug monitoring (TDM) TEa criteria are: Ricos et al., or 1/6th of the reference (therapeutic) interval CVa data State-of-the-art Summary IQC rule selection can proceed via Statistics TEa and statistics  Development of an IQC policy has to consider both possibilities. Conclusion Because regulation and manufacturer recommendations describe minimum IQC, only, the laboratory has to develop an IQC policy itself. The IQC-policy of the laboratory has to respect statistical and medical aspects and should be developed according to the needs of the laboratory, naturally, respecting the guiding rules of regulation. Development of IQC policy  Laboratory task Slide 22 Statistics & graphics for the laboratory 75 IQC policy Introduction Software Samples Frequency (& location) of IQC measurements Performance (State-of-the-art) IQC rule selection –Patient release –Process control –Examples IQC rules for state-of-the-art performance –Screening with TEa/CV a,tot –STT IQC decision tool –EZ rules/Validator Special topics –Calculation of an actual TE[a] –Rule “n” and patient release –Dealing with a bias –IQC rules with wide limits (e.g. 6s) and lot variations –“Fine-tuning” of IQC according to instability Remedial actions & –P fr of the IQC rule and frequency of remedial actions –CV a,tot /CV a,w ratio –Inspecting IQC charts Summary Slide 23 Statistics & graphics for the laboratory 76 IQC policy Introduction The IQC policy will be developed on the basis of the Belgian situation: RECALL: Koninklijk Besluit Art. 34. §1. The laboratory director has to organize IQC in all disciplines. §3. IQC consists of several procedures which allow, before the release of patient results, to detect all significant within- or between-day variations: Remedial actions policy/rule selection Art. 35. §1. The frequency of control measurements has to be such that it can guarantee a clinically acceptable imprecision. This frequency depends on the characteristics of the method and/or the instrument: Additional: Praktijkrichtlijn Control rules used for start and for acceptance of a run  at least 2 IQC events. Rule selection/frequency §2. The control material, … must be stable within a defined period of time. Different aliquots of the same lot must be homogeneous; Concentrations: Praktijkrichtlijn. Sample requirements §3. For each new lot, the mean and the SD have to be determined. … IQC materials may, at the same time, not be used as calibrator and control material: Establish stable performance: own targets! Slide 24 Statistics & graphics for the laboratory 77 IQC policy Introduction On the basis of the Belgian regulation, we will apply all input elements that we have seen to the analytes ISE; substrates; enzymes, specific proteins; TDM. The elements we have seen (see also “ckecklists”) were: “Knowledge-base” -Statistics -Rule selection -Metrology/Error concept for IQC -TEa data, biological variation State-of-the-art performance data -CV a,tot, CV a,w -Experience from "Peer-IQC" -Laboratory experience (also: Questionnaire to participants) Software tools -Laboratory IQC software -EZ-rules -IQC decision tool/Ratio TEa/CV a,tot From established rules, we look at the magnitude of undetected errors (e.g., in 10% of the cases)  Additionally, we will address the topic -Remedial actions. We will develop the policy in an interactive way. We start with the Laboratory IQC software (see also checklist). Slide 25 Statistics & graphics for the laboratory 78 IQC policy IQC-software Discussion ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… …………………………………………………………………………………….………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… …………………………………………………………………………………… Slide 26 Statistics & graphics for the laboratory 79 IQC policy The IQC sample Samples: §2. The control material, … must be stable within a defined period of time. Different aliquots of the same lot must be homogeneous. Concentrations: Praktijkrichtlijn  See also checklist – Samples  Select commercial one Note: In principle, their target and range cannot be used by the laboratory (Art. 35. §3.) (§3. For each new lot, the mean and the SD have to be determined). Consider participation in an IQC "Peer"-system Sample advantages due to the high number of participants System specific target means and SDs Low target uncertainty Control of sample stability Discussion ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… …………………………………………………………………………………….………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ………………………………………………………………………………………… Slide 27 Statistics & graphics for the laboratory 80 IQC policy Frequency of measurement and location Frequency: Art. 35. §1. The frequency of control measurements has to be such that it can guarantee a clinically acceptable imprecision. This frequency depends on the characteristics of the method and/or the instrument. Praktijkrichtlijn Control rules used for start and for acceptance of a run.  Frequency: at least 2 IQC events  Location: at least at start and end  See also checklist – Frequency and location Discussion ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… …………………………………………………………………………………….………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ………………… Slide 28 Statistics & graphics for the laboratory 81 IQC policy Stable performance §3. For each new lot, the mean and the SD have to be determined.  The laboratory has to establish its own stable performance data. REMEMBER The target SD (= stable imprecision) is the cornerstone of IQC. It deserves special attention. All instabilities (random and systematic) are compared relative to the stable imprecision. Recommendation 1 Compare your performance with your colleagues (IQC-"Peer") Advantages due to the high number of participants Better IQC-sample Easier set-up of IQC (more reliable estimates of stable performance) Easier troubleshooting by direct comparison with “peer” Recommendation 2 Compare your performance with the data from the manufacturer! See EXCEL-file for an overview about “state-of-the-art” performance data for the Analyte groups Ion selective electrode (ISE) Substrates/Enzymes Specific proteins Therapeutic drug monitoring (TDM)  A list of total and within-run CV data for analyte concentrations in the reference interval (RI) can be found in the EXCEL-file (sorted by biological CVw).  See also Checklist “stable” imprecision Note: CV data from concentrations in the RI have been chosen because they will be compared with TEa goals derived from biological variation, which apply to concentrations within the RI. Slide 29 Statistics & graphics for the laboratory 82 IQC policy Stable performance Discussion ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… …………………………………………………………………………………….………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… …………………………………………………………………………………… Slide 30 Statistics & graphics for the laboratory 83 IQC policy IQC rule selection Generally, IQC rule selection can be done on: Statistical basis: A rule is chosen based on P fr and P ed. SD-limits are taken from stable performance. TEa basis: From a specification for TEa, critical error values can be calculated. From the critical error values, adequate IQC rules can be selected, naturally, on statistical basis. Selection tools are: power functions, OPSpecs, critical error graphs, the TEa/CV a,tot ratio; the IQC decision tool  See also checklists basic statistics; power of control rules; metrology. Belgian regulation, however … §3. IQC consists of several procedures which allow, before the release of patient results, to detect all significant within- or between-day variations Art. 35. §1. … guarantee clinically acceptable imprecision? What is significant; what is clinically acceptable?  We need a goal for TEa! … requires, in principle, IQC rule selection on the TEa basis.  see also TEa checklist Slide 31 Statistics & graphics for the laboratory 84 Selection of TEa Discussion ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… …………………………………………………………………………………….………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… …………………………………………………………………………………… IQC policy Slide 32 Statistics & graphics for the laboratory 85 IQC policy Example Selection of the TEa  We select TEa from biology TEa from the Ricos concept, and The most stringent TEa, with bottom-line  We apply TEa to manufacturer’s performance Example Serum Glucose Ricos TEa: 6.3% CV a,tot : 2% Bias: 0% Exercise STT IQC decision tool EZ Rules selection: NONE with 90%: see OPSpec (note TEa/CV a,tot = 3.2) Conclusion from the example: It did not work!  We need a “modified TEa approach” Slide 33 Statistics & graphics for the laboratory 86 Apply the TEa approach (TEa/CV a,tot ratio; the Validator®/EZ rules; IQC decision tool) as evaluation tool for current quality. Analytes that cannot fulfill the TE specifications  Decide on the most stringent rule you want to apply (P fr ; n) Analytes that fulfill TE very easy  Decide on the most loose rule you want to apply All others: decide whether you want to use individually optimized rules: –Different rules –Same rule, but movement –Adapt the frequency (less/more measurements) Problem Optimization always should consider individual test stability! Example 1: most stringent rule (left) Westgard multirule: 1 3s / 2 2s / R 4s / 4 1s n = 4 (required for full power) P fr : 3%  Relatively high! P ed for a 2.5s-shift ~90% P ed for 3 RE ~90% Example: the most loose rule (right) 1 3.5s n = 4 P fr : 0,2%; P ed for a 2.5s-shift ~50% P ed for 3 RE ~70% NOTE For what concerns n, the rule can be applied flexible (e.g., n = 1). IQC policy The modified TEa approach Mean 3.5 s Slide 34 Statistics & graphics for the laboratory 87 Example 2 for the most stringent rule (left) Westgard multirule: 1 3s / 2 2s n = 4 P fr : 1% P ed for a 2.5s-shift ~83% P ed for 3 RE ~83% Example for the most loose rule (right) 1 3.5s n = 4 P fr : 0,2%; P ed for a 2.5s-shift ~50% P ed for 3 RE ~70% NOTE For what concerns n, the rule can be applied flexible (e.g., n = 1). Note (most stringent rule): The Westgard multirules have been chosen because of their widespread availability in IQC software programmes. When possible, choose others: Think of control rules with variable limits, or mean and variance rules! We will go further with The modified TEa approach IQC policy The modified TEa approach 1 3s /2 2s 3.5 s 1 3s /2 2s 3.5 s Slide 35 Statistics & graphics for the laboratory 88 IQC policy IQC rule selection – Process control Basic idea Independent of using IQC for patient release, we may want to use it for process control!  For process control, each test could be controlled by the same IQC procedure.  For process control, higher P fr values can be accepted. However, P ed should be considerably higher than for patient release. Examples Westgard multirules with high n P ed (90%) ~1.8  SE (see Figure); for the logic of Westgard multirules: lower Figure Mean and variance rules with high n Logic scheme for applying the Westgard multirule X Control Data 1 2s 1 3s 2 2s R 4s 4 1s 10 In-controlAccept run Out-of-controlReject run No Yes No Yes No 1 2s /1 3s /2 2s /R 4s /4 1s /10 x(mean) Pfr: ~3%, Ped for a 2.5s shift: 90% (for n = 4, without the mean rule) Slide 36 Statistics & graphics for the laboratory 89 IQC policy Rule selection Discussion ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… …………………………………………………………………………………….………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… …………………………………………………………………………………… Slide 37 Statistics & graphics for the laboratory 90 IQC policy Selection of IQC rules – Tools & Examples Tools The ratio of TEa/total-CVa: should be  4 The STT IQC decision tool The Westgard software tools (Validator/EZ Rules) EXERCISES Sodium We apply the modified TEa approach, the “most stringent TE” (with bottom-line), and the state-of-the-art CV data 1. Select TEa: Bottom-line: 3.2% 2. Apply the IQC decision tool - Set stable Bias = 0 - Stable CV (total) = 1% 3. “Sort of analyte?” 4. Apply Validator® for individual rule selection OUTCOME The IQC tool and the Validator® show that sodium, with the values chosen, cannot be controlled according to the TEa concept. The ratio of TEa/total-CVa is 3.2. Triglycerides Strategy as before (see Figures on the right) TEa: 13.8 (most stringent) Bias: 0% CV a,tot : 1.7% Ratio TEa/CV a,tot : 8.1 OUTCOME The IQC tool and the Validator® show that, even with the most stringent TEa, triglycerides can be controlled easily with the TEa concept:  The Validator® selects the 1 3.5s rule (n = 2) NOTE In the EXCEL-file, you find for ISE, substrates, enzymes, specific proteins, TDM a list of CVw, CVb TE for IQC decision - Ricos et al. State-of-the-art CV data Slide 38 Statistics & graphics for the laboratory 91 IQC policy IQC rule selection – Applications First, we screen the analytes with the Ricos-TEa/CV a,tot ratio The tables show analytes with ratios Slide 39 Statistics & graphics for the laboratory 92 IQC policy IQC rule selection – Applications The tables below show analytes with ratios  4 (full list in EXCEL-file).  In principle, individually optimized IQC rules can be applied for all of these analytes (note: most loose = 1 3.5s ). Now, we will apply the STT IQC decision tool for some analytes with ratios from 4-7. APO A1 (4,3):TEa: 9.1CV a,tot : 2.1  -1-Antitrypsin (5,1)TEa: 9.2CV a,tot : 1.8 IgA (6,8)TEa: 13.5CV a,tot : 2.0  We observe: from ratios >~5-6, IQC should be relatively easy!  We verify this with the EZ rules (note: fill in all fields; if appropriate, use dummy values). TEa/CV a,tot ratio Slide 40 Statistics & graphics for the laboratory 93 IQC policy IQC rule selection – Applications Excercises with the EZ rules, using the specific protein examples (ratios from ~4-7). Note: all fields must be filled in (may be dummy values)! Selection criteria: 2 materials, SE detection 90% APO A1 (4.3)Rule: 1 2.5s n=4 P fr : 4%  -1-Antitrypsin (5.1)Rule: 1 3s / 2 2s n=2 P fr : 1% IgA (6.8)Rule: 1 3.5s n=2 P fr :