Slide 12014 CTL PTP WORKSHOP Ingrid Flemming IFM Quality Services Pty Ltd Slide 2Today: The official agenda: Outcomes and summary from recent PTPs – Touch Current – Heating transformers – Heating switches – Cables, conductor resistance and insulation thickness A walk down the road……… – Learning from Histories – The guilty and the innocent – Effectiveness testing Slide 3Today: The unofficial (but actual) agenda – Starting fires – Fighting fires – Resolving risk Slide 4Raise hands Who attended the 2013 CTL PTP Workshop? Who took something of benefit home with them? Who shared the workshop content with their co- workers? Whose laboratories changed ANY procedure as a result of attending the workshop? How will we get MORE from attending the workshop? Slide 5September 2013, was NSW’s hottest and driest September since the beginning of weather record keeping. In October 2013, NSW was on fire. Slide 6Slide 7Slide 82 people lost their lives The damage costs were enormous. Many insurance policies will not pay. Slide 9Then... In the face of devastation Starts the “ BLAME GAME ”. – The green (political) party is at fault because back burning was legislated out – The government has done nothing to mitigate climate change – Local authorities (that approve building work) allowed us to build here – The parents of those naughty children that started fires should have minded their kids. Slide 10We were not told how to prepare our house – The staff did not know how to perform the test We didn’t know how to use the fire equipment – The staff was not trained to use the equipment We didn’t know there was a ban on lighting fires outside at this time – We didn’t know we should apply the standard Slide 11Slide 12Slide 13Conformity and non-conformity We all understand conformity Do we all understand non-conformity? We all know there is a COST of Conformance What about cost of non-conformance? We are coming back to this........... Slide 1413E36 - TOUCH CURRENT Slide 15Slide 16Slide 17Slide 18Slide 19M=0.48, 77 outliers, sinusoidal M=0.3, 139 outliers non-sinusoidal M=7.5, 64 reported low results, sinusoidal M=0, 82 “outliers”, sinusoidal M=3.5, 73 outliers, wave form mixed reply Slide 20Small resistors Point 3 was expected to yield a very high result (7.5 mA, 240 V input) 64 laboratories reported a measurement less than 3.5 mA, including >20 “0” results. An additional 24 laboratories reported the words “pass” as the result. 88 laboratories would have passed an unsafe sample. Slide 21Slide 22Point 4 was not connected to any part of the circuit. (A rivet attached to the plastic box.) 193 laboratories reported a measurement greater than zero, up to 0.05 mA. 24 laboratories reported greater than 0.05 mA. 8 laboratories reported greater than 3.5 mA. Slide 23Slide 24The forest has burned It is too late to call the fire brigade There is no-one else left to blame Slide 25It is time for renewal Slide 26Discussion point: What kind of renewal is needed for our laboratories? Slide 27Outcomes: renewal discussion Slide 28Bringing renewal about From where does change need to originate? Technical staff? Supervisors? Lab managers? Company heads? ISO/IEC 17025 4.2.3 Top management shall provide evidence of commitment to the development and implementation of the management system and to continually improving its effectiveness. Slide 29Slide 30Slide 31Slide 32Equipment 30-40% additional labs are required to perform follow up activities if equipment audits occurred. The audits issued by IFM receive “outliers” ONLY when the reported measurement is contrary to the requirements in the standard. Test staff perform the audit, but nobody seems to CHECK the audit results – EXCEPT IFM. Why do labs wait for IFM to tell them to comply with the standard? Slide 33The IFM equipment audits relating to PTP are NOT comprehensive. Equipment performance needs to be verified and/or validated – IFM cannot “audit” this Items such as steadiness of power supply, also need to be checked. Who does this? Who will believe the lab checks equipment performance if the PTP audits show labs do not get the basic equipment questions right? Slide 34ISO/IEC 17025 5.5 5.5.2 Equipment and its software shall be capable of meeting the accuracy required and shall comply with specifications...... Before being placed into service, equipment shall be calibrated or checked to see it.... complies with the relevant standard specifications.... It shall be checked and/or calibrated before use. Slide 355.5.3 Equipment shall be operated by authorised personnel 5.5.5c records shall be maintained of.... Checks that equipment complies with the specification Slide 365.2 Personnel 5.2.1 the laboratory management shall ensure the COMPETENCE of all who (perform tasks)....Personnel performing specific tasks shall be qualified on the basis of appropriate education, training, experience and/or demonstrated skills.... 5.2.2...... The laboratory shall have a policy for identifying needs and providing training..... THE EFFECTIVENESS OF TRAINING ACTIONS SHALL BE EVALUATED. Slide 37Discussion: What does it mean to train someone? General education on concepts Pass on skill and knowledge Make information easily available and keep it up to date Scope of training to be defined (depth of understanding vs monkey-work) Mentorship – Explanation of procedure – Demonstration/observation – Works under supervision – Effectiveness of training assessment, discussion and demonstration – Approval for testing – Still OK after time? Provide info for the trainee to absorb More intensive obtaining of info, not to forget Assessment of both knowledge and skills Behaviour of the trainee – observation Slide 3813E33 Temperature Rise in Switches Slide 3913e33 Test method Attach conductors between switch and power source Attach thermocouples to specified part of switch Mount the switch on wooden surface Apply suitable specified current. Monitor until steady state has been reached (as stated in ALL standards quoted) Determine the temperature rise Variation – on one of the samples, perform additional actuations while “live”. This was only to demonstrate whether initial actuations were adequate to clear manufacturing residues from switch. Slide 40Questions relating to 13e33 I would like to ask more detail with the testing method of DC supply, we have no DC power source with high power like 12A and we just only have 3A with low voltage. So can we skip the part of the DC measurement and providing with AC? Answer: it is specified in instructions only to apply tests in scope Slide 41So Which program (part01 or part 02) should I enter the result at online system ? Answer: Instructions contain information about part01 and part02 Slide 4217025 section 4.4 Review of requests, tenders and contracts... – 4.4.1.a ensure that the customer’s requirements are understood, including methods to be used Concept: In PTP – the provider is the customer Slide 43I setup my test circuit today with the test switches installed. Something is wrong. Neither one of the switches are working. They do not allow current to pass. I checked the switches when we received them by performing 20 mechanical operations and they were fine. The test results are due in 17 days. Answer: run current through the switch according to the instructions, not across the insulation. Slide 4413E33 OUTCOMES Slide 4513e33 Temperature Rise - Switches Slide 46Group Exercise Identify the aspects of training that would be required for staff to set up a heating test in switches How will effectiveness of training be determined? How will the performance of all staff doing this test be monitored? Slide 47Training? Standard Procedure Thermocouples: preparation, selection, fixing, attachment, positioning Load Connection Control of environmental conditions Power supply Installation Time Equip specification Calibration Using equip Personal safety Slide 48Ishikawa Problem Measurement MaterialsMethodsEquipment EnvironmentPeople Slide 49Problem Measurement MaterialsMethodsEquipment EnvironmentPeople Slide 50Slide 51Temp Rise Switches PersonEnvironment Equipment & Materials MeasurementsProcedure Slide 52(2) We did not understand CTL DS 335 about steady state (1) Requesting Temperature rise in Kelvin was a dirty trick! (1) We did the wrong test (2) We did not understand CTL DS 335 about steady state (1) Requesting Temperature rise in Kelvin was a dirty trick! (1) We did the wrong test Training Supervision Competence Skill Ability to Think and Observe Slide 53(1) The power supply was under the test surface and contributed to heating of switch (2) Air conditioner was in direct line of switch (1) The power supply was under the test surface and contributed to heating of switch (2) Air conditioner was in direct line of switch Control of Ambient Influence of external factors or equipment Slide 54(1) Accidental use of wrong thermocouple (T not K) (1) Wires were old and worn (2) Wrong current source used (1) Inappropriate Rectifier used (2) Monitoring device not calibrated (1) Accidental use of wrong thermocouple (T not K) (1) Wires were old and worn (2) Wrong current source used (1) Inappropriate Rectifier used (2) Monitoring device not calibrated Regulated power source Current meter Thermocouples Soldering equip Calibration Good repair Slide 55Procedure (6) Additional actuations removed residue from switch (1) TC attached without copper discs makes a difference (1) Current applied suddenly vs gradually (3) Burnt switch with soldering iron (6) TC attached incorrectly/inappropriately (1) TC attached in wrong position (2) conductor / meter connections inadequate (1) applied wrong ambient test temperature (6) Additional actuations removed residue from switch (1) TC attached without copper discs makes a difference (1) Current applied suddenly vs gradually (3) Burnt switch with soldering iron (6) TC attached incorrectly/inappropriately (1) TC attached in wrong position (2) conductor / meter connections inadequate (1) applied wrong ambient test temperature Correct sample preparation (actuations, clean terminals etc) Correct sample connection (conductors, thermocouples, monitoring devices) Slide 56..... Results in Kelvin is a dirty trick! ??..... We didn’t know when we had reached steady state... ???..... Results in Kelvin is a dirty trick! ??..... We didn’t know when we had reached steady state... ??? Interpretation Units Calculation Observation Understanding Interference Slide 57One misfit Slide 58Group exercise Given the problems that were identified for 13e33, what changes / additions are required for the training and monitoring needs of staff identified in the last exercise? See next: Slide 59(2) We did not understand CTL DS 335 about steady state (1) Requesting Temperature rise in Kelvin was a dirty trick! (1) We did the wrong test (1) The power supply was under the test surface and contributed to heating of switch (2) Air conditioner was in direct line of switch (1) Accidental use of wrong thermocouple (T not K) (1) Wires were old and worn (2) Wrong current source used (1) Inappropriate Rectifier used (2) Monitoring device not calibrated (6) Additional actuations removed residue from switch (1) TC attached without copper discs makes a difference (1) Current applied suddenly vs gradually (3) Burnt switch with soldering iron (but did the test anyway) (6) TC attached incorrectly/inappropriately (1) TC attached in wrong position (2) conductor / meter connections inadequate (1) applied wrong ambient test temperature Slide 60Additions to training on heating test Slide 61Whole group exercise Pick any of the “causes” identified earlier and perform 5 why analysis to obtain a theoretical root cause (Theoretical, because we have no other details apart from what is displayed.) Slide 6217025: 5.9 Assuring the quality of test and calibration results 5.9.1 The laboratory shall have quality control procedures for monitoring the validity of tests and calibrations undertaken. The resulting data shall be recorded in such a way that trends are detectable and, where practicable, statistical techniques shall be applied to the reviewing of the results. This monitoring shall be planned and reviewed and may include, but not be limited to, the following: a) regular use of certified reference materials and/or internal quality control using secondary reference materials; b) participation in interlaboratory comparison or proficiency-testing programmes; c) replicate tests or calibrations using the same or different methods; d) retesting or recalibration of retained items; e) correlation of results for different characteristics of an item. NOTE The selected methods should be appropriate for the type and volume of the work undertaken. Slide 635.9.2 Quality control data shall be analysed and, where they are found to be outside pre- defined criteria, planned action shall be taken to correct the problem and to prevent incorrect results from being reported. Slide 64The cost of non-conformity Failure to “get it right the first time”, eventually costs at least 3 times more than doing it right. The initial time spent doing it wrong is wasted Time is spent, normally by more than the original staff determining what happened The time repeating the task is lost opportunity to achieve something else Slide 6513E35 Cables Conductor Resistance and Insulation Thickness Slide 6613e35 Cables PTP Slide 67Slide 68Slide 69Slide 70Slide 71Slide 72The point about insulation thickness test Multiple measurements are made routinely Lends itself to quality activities without any additional effort or real cost Retained samples can be used to report repeated measurements on “same” cable at certain cyclic times when the test is conducted. Alternatively, the standard error or %CV can be plotted in a control chart to monitor the consistency of measurements. Either of the above has little additional cost, but adds value to the confidence of the lab and its management. Slide 73Presentation by Udo Krischke, SGS Quality Control in Analytical Chemistry Slide 74Dr. Udo Krischke ETF12 Convener CTL Meeting PT Workshop Matsue, 14.05.2014 Slide 75Outline No comprehensive training Share good practice from analytical chemistry In line with ISO 17025 §5.9 requirements REFERENCE SAMPLES CONTROL CHARTS Slide 76Reference Samples – What are they? Reference Samples are ‘controls’ or standards used to check the quality and traceability of test results. A reference standard for a unit of measurement is an artifact that embodies the quantity of interest in a way that ties its value to the reference base for calibration. At the highest level, a primary reference standard is assigned a value by direct comparison with the Standard (metrology). A primary standard is usually under jurisdiction of a national standards body. – For example, mass is defined by an artifact maintained by the Bureau International des Poids et Mesures in Sèvres, France. Since most analytical instrumentation is comparative, it requires a sample with known properties for accurate calibration. These samples are produced under stringent manufacturing procedures and differ from laboratory samples in their certification and the traceability of the data. Slide 77Reference Samples – Quality Levels Certified Reference Material (CRM) – A specially manufactured and commercial available Reference Material, accompanied by a certificate, one or more of whose property values are certified by a procedure which establishes its traceability to an accurate realisation of the unit in which the property values are expressed and for which each certified value is accompanied by an uncertainty. Reference Material (RM) – A purchased material or substance one or more of whose property values are sufficiently stable over time, homogeneous and well established to be used for the calibration of an apparatus, the assessment of a measurement method, or for assigning values to materials. In-house Standards – A material or substance which fulfils the quality criteria of RM but has its origin from a “real life” sample which is well enough studied to understand its relevant properties to act as a RM. Slide 78Reference Samples – What‘s next? Use the RM to check your instruments Compare resulting values with expected values Judge on results based on an established acceptance or refusal protocol Act according to this document. Done? What does this single result at a given time tell you about the performance of your instrument? Slide 79Control Charts Chart for QC samples, including reagent blanks, laboratory control samples, calibration check standards, etc OVER TIME Upper Control Limit X + 3s Upper Warning Limit X + 2s Lower Warning Limit X - 2s Lower Control Limit X - 3s Average Slide 80Control Charts - Control Chart for Means Constructed from the average and standard deviation of a specified number of measurements of the parameter of interest. The accuracy chart includes upper and lower Warning Levels (WL) and upper and lower Control Levels (CL). Common practice is to use 2s and 3s limits for the WL and CL, respectively, where s may represent the standard deviation of the respective pre-control period. Slide 81Practical Example of Control Charts – Cadmium in Plastic CRM acc. IEC 62321 Certified Reference Material 680, Cd content = 140.8 mg/kg Certified Reference Material 681, Cd content = 21.7 mg/kg Expected value Mean value LWL LCL UCL UWL Slide 82Control Charts – Out of Control Situations Figure 3 Control Limit Warning Limit Average Slide 83Control Charts – Out of Control Situations If the warning limits (WL) are at the 95% confidence level; statistically – 1 out of 20 points would exceed that WL – and only 1 out of 100 would exceed the CL Control limit – one measurement exceeds a CL Warning limit – two out of three successive points exceed a WL One point fall into the WL at one side, and the next point goes to the WL on the other side Four out of five successive points exceed 1s, or are in decreasing or increasing order Seven successive samples are on the same side of the average line Seven successive point shows the same trend (up or down) Slide 84Treatment of Out-of-Control Situations Create a procedure to deal with Out-of- Control Situations and act according to it! Take corrective action promptly to determine and eliminate the source of error Do not report data until the cause of the problem identified and corrected ? Slide 85Treatment of Out-of-Control Situations Maintain records of all out-of-control events, determined causes, and corrective action taken Involve lab and/or quality management in the clearing process of Out-of-Control Situations Establish preventive actions: not only to eliminate such events, but also to reduce repetition of the causes – Identify and clear root causes of systematic errors Slide 86Summary ISO 17025 §5.9 asks for quality control procedures to monitor and statistically evaluate the validity of test results and calibrations Reference samples and control charts are powerful tools for everyday QA/QC in a lab Provide answers to the current state of test equipment but even before out-of-control situations may happen Slide 87Thank you for your attention Open questions? Slide 8813E32 Outcomes Slide 89Initial questions from participants 13e32 RE: Sample Preparation and Mounting, it says Attach thermocouples to blackened copper or brass discs that are 1 mm thick and 15 mm in diameter. The discs should be placed such that they are flush with the surface of the plywood support. My question: What is the purpose to prepare the disc? How to place it? Could you please show us one picture on this disc? Slide 90Our lab is taking part in the IFM Electrical Proficiency Testing Program 13e32 Temperature Rise-Transformers these days. Because the different language we using, we got some communications difficulties about the program, and a question for you. 1,In order to achieve the SEC1 and SEC2 output current specified in Table 1. e.g.. for 13e32 General, both of SEC1 and SEC2 output current should be 1.0A initially, and the resistance rise with the time, so the current should not be the initial current. Do we need to keep the initial current 1.0A by changing load resistance during the test till the end or keep the resistance and let the current drop or rise. Slide 91The instructions I’ve received do not state to load transformer with an impedance however 613558-1 clause 14.1 states: ‘Transformers are supplied…. And loaded with an impedance producing the rated output” Can you confirm for me if I should be loading the transformer from the start of the test ? Slide 92We are preparing to conduct the tests for 13e32 Temperature Rise - Transformers and clarification is needed on the sample. Please confirm whether this is a Class 1 transformer. Also, are the secondaries individual or series parallel? Slide 93Item 5 of Sample Preparation and Mounting in your instructions said: Select wires approximately 150cm in length, with a nominal cross-sectional area of 1.5mm2. Connect wires to primary PRI and secondary windings SEC1 and SEC2. Our question: Shall we consider the resistance of wires during the test? Slide 94We have received sample and instruction for PTP in the subject. I have a question about test set-up that I think is not well explained and may be interesting to other participants to the program: The primary circuit (PRI) of the transformer is clear to be connected to the input voltage of 240V, 50Hz and 60Hz; but about secondary windings (SEC1 and SEC2), in order to obtain the specified currents, (1A+1A for general and 0.9A+1.2A for 60335-1) which kind of connection is needed? Have each one to be short circuited? … Or a test load shall be applied at SEC1 and SEC2? In the last case, which kind of test load is required (resistive, inductive or mixed)? Slide 95Answer, according to the standard To obtain particular current in SEC 1 and SEC 2 load has to be connected. As there is not mention any cos fi load it is assumed that it should be resistive load.. Slide 96The PT participants who are conducting a Temperature Rise test on a transformer are currently encountering challenges with regards to proceeding with the test. The reason being that they do not know what is the required resistive load to be used for this test, that should be connected to the secondary side of the transformer? Answer: the load is the one that provides the required output. Slide 97Are we supposed to purchase a relay for the PTP test with the transformer? The instructions did not state that. Slide 98Please check we have the correct configuration Don’t worry about coin currency, we are obtaining US 1c coins and will use these instead of our local currency. Slide 99Slide 100Group exercise Based on the requirements for a heating test in transformers, determine the training needs of staff for performance of this test. How will the effectiveness of training be determined? How will ongoing monitoring be conducted for all staff performing such tests in the laboratory? Slide 10113e32 Temperature Rise - Transformers Slide 102Slide 103Slide 104Slide 105Slide 106Slide 107For transformer heating tests What would you say are the MOST critical items? ? Correctly loading secondary winding(s) ? Reaching steady state? ? Rapid measurement of resistance after disconnect ? Equipment can “manage” the rapid readings ? Reading for sufficient time after disconnect to ensure the curve is right? Slide 108Examine the following data (take notes, please) – Timing of the first readings after disconnect? – Time intervals between readings? – Duration that readings were performed – Total number of readings Slide 109Example 1 Slide 110Example 2 Slide 111Example 3 Slide 112Example 4 Slide 113Example 5 Slide 114R at T=0 was 272 ohm Example 6 Slide 115R = 276 Example 7 Slide 116Discuss For the previous slides What similarities and differences were noted with timing of the first readings? Time intervals between readings? Duration that readings were performed Total number of readings – Is the MU affected by the above variables? – For CTL document 251D, should we be specifying only equipment accuracy for this test, or should we be including its performance/capability to measure values for heating test? (It might be accurate, but only good for static readings.) Slide 117Temp Rise Transformers PersonEnvironment Equipment & Materials MeasurementsProcedure Slide 118(4) Typos/careless mistakes/calculation (6) did not understand 60335-1 and/or training in standard (3) poor training other than above (1) lack of supervision or oversight (4) Typos/careless mistakes/calculation (6) did not understand 60335-1 and/or training in standard (3) poor training other than above (1) lack of supervision or oversight Training Supervision Competence Skill Ability to Think and Observe Slide 119(2) Ventilation directly affecting sample (1) Bad control of ambient temp (2) Ventilation directly affecting sample (1) Bad control of ambient temp Control of Ambient Influence of external factors or equipment Slide 120(3) Ohm meter capabilities (Too much time to stabilise Not suitable) (2) Not functioning to spec (2) quality of electrical connection (3) Ohm meter capabilities (Too much time to stabilise Not suitable) (2) Not functioning to spec (2) quality of electrical connection Regulated power source Volt / Current meter Resistive load Thermocouples Soldering equip? Calibration Good repair Slide 121Procedure (8) did not place sample on copper discs to obtain max temp (3) copper discs not flush with surface (1) did not follow internal procedure (2) did not wait for steady state (2) wrong procedure followed (1) windings separately loaded (1) incorrect testing set-up (8) did not place sample on copper discs to obtain max temp (3) copper discs not flush with surface (1) did not follow internal procedure (2) did not wait for steady state (2) wrong procedure followed (1) windings separately loaded (1) incorrect testing set-up Correct sample preparation Correct sample connection (conductors, thermocouples, monitoring devices) Slide 122(3) Choice of equipment inappropriate (3) Interpolation or extrapolation of resistance data (1) Did not correctly read result (3) Choice of equipment inappropriate (3) Interpolation or extrapolation of resistance data (1) Did not correctly read result Interpretation Units Calculation Observation Understanding Interference Slide 123Are there common threads to causes? Slide 124Evaluate this corrective action – This is already 2 nd attempt for submission. (Lab was asked to propose preventive actions.) According to the participation in proficiency test program 13e32 temperature rise of transformer, most of our test results are outlier from the acceptable range. See details in results report of this program Slide 125Root cause analysis We installed the transformer on a test corner inside a test room. We also used additional wires connected to the transformer instead of measuring the resistance directly from the transformer. Slide 126RCA cont. 1) The testing room is verified for air-flow at the test area every 6 months. Prior to conduct the proficiency test, the airflow was within the limit as 0.5m/s. After the proficiency test results came out, we re-verified the air-flow in the test room if this was the factor affecting the test. There was some problem at the air filter so the air-flow was higher than 0.5 m/s. 2) The additional wires connection may affect the winding resistance measurement. The total resistance for each winding was different due to the additional wires. Slide 127Corrective action 1) Reparation of air filter and re-verification of air flow and the velocity is less than 0.5 m/s. Then re-testing is conducted. 2) The winding resistance measurement is measured as close as the transformer terminals. Preventive action Reduce the verification period as every month to ensure the air-flow does not exceed the limit Did the laboratory get to the root cause? Do the actions address the root cause? Slide 128LEARNING FROM HISTORIES Big picture Slide 129Take this down....... O = “opened” something was entered in IFM’s system X = reminder sent by IFM A = acknowledgement received from lab N = IFM notified lab about something R = response received from lab, but not enough to close S = status (summary for management) C = item was closed F = item was referred to IECEE as over due Slide 130Group exercise From the handout and from each laboratory (separately), look at the participation history. Determine performance over time (tally in 4 time frames) – before 2008, – 2009-2011, – 2012-2013 – After 2013 What are the number of outliers in each time frame? (success rate%) Are there potential problem topics? In your opinion, should the lab focus on any particular topic? Are there common threads with respect to reasons for outliers which the lab should pay attention to? Are there other recommendations? Slide 131Temp. Rise - Transformers05e24200510 13e32201307 Touch Current09e36200907 O6/01/2010Frequency range of instrument calibrations is questioned. R21/01/2010laboratory requested clarification on the nature of the outlier. C28/01/2010revised calibration certificate received. Slide 132Summarising findings from examples Slide 133Is there a root cause for the large number of PTP outliers? Slide 134Finally How will we embark on this activity of renewal? Slide 135