Discussion on DUT Uncertainty at CCM-WGFF Chun-Min Su

Скачать презентацию Discussion on DUT Uncertainty at CCM-WGFF Chun-Min Su

1207e4262aa51738c399c833154dc0fc.ppt

Количество слайдов: 24

Discussion on DUT Uncertainty at CCM-WGFF Chun-Min Su CMS/ITRI, Chinese Taipei

Background (1) • Paragraph N 5 of MRA-D-04 says “Contributions to the uncertainty stated on the calibration certificate include the measured performance of the device under test during its calibration at the NMI or accredited laboratory. CMC uncertainty statements anticipate this situation by incorporating agreed-upon values for the best existing devices. ” BED Copyright 2012 ITRI 業技術研究院

Background (2) • Recommendations in DOCUMENT JCRB-8/9 “Uncertainty contributions of the device under calibration or measurement” by Ad hoc JCRB Working Group CMC Uncertainties: 1. It is recommended to exclude contributions to the CMC uncertainty caused by the customer device before or after its calibration or measurement at the institute. 2. It is recommended to include contributions to the CMC uncertainty caused by the best ordinarily available customer device during its calibration or measurement at the institute. In general, these values will be published in Appendix C of the CIPM MRA. For individual certificates the actual characteristics of the customer device must be considered. Copyright 2012 ITRI 業技術研究院

Background (3) ILAC-P 14: 12/2010 • ILAC Policy for Uncertainty in Cal. (Clause 5. 4) – Calibration laboratories shall provide evidence that they can provide calibrations to customers in compliance with 5. 1 b) so that measurement uncertainties equal those covered by the CMC. In the formulation of CMC, laboratories shall take notice of the performance of the “best existing device” which is available for a specific category of calibrations. – A reasonable amount of contribution to uncertainty from repeatability shall be included and contributions due to reproducibility should be included in the CMC uncertainty component, when available. There should, on the other hand, be no significant contribution to the CMC uncertainty component attributable to physical effects that can be ascribed to imperfections of even the best existing device under calibration or measurement. Copyright 2012 ITRI 業技術研究院

Background (4) ILAC-P 14: 12/2010 – It is recognized that for some calibrations a “best existing device” does not exist and/or contributions to the uncertainty attributed to the device significantly affect the uncertainty. If such contributions to uncertainty from the device can be separated from other contributions, then the contributions from the device may be excluded from the CMC statement. For such a case, however, the scope of accreditation shall clearly identify that the contributions to the uncertainty from the device are not included. – NOTE: The term “best existing device” is understood as a device to be calibrated that is commercially or otherwise available for customers, even if it has a special performance (stability) or has a long history of calibration. Copyright 2012 ITRI 業技術研究院

Background (5) ILAC-P 14: 12/2010 • ILAC Policy for Uncertainty in Calibration (Clause 6. 4) – Contributions to the uncertainty stated on the calibration certificate shall include relevant short-term contributions during calibration and contributions that can reasonably be attributed to the customer’s device. Where applicable the uncertainty shall cover the same contributions to uncertainty that were included in evaluation of the CMC uncertainty component, except that uncertainty components evaluated for the best existing device shall be replaced with those of the customer’s device. Therefore, reported uncertainties tend to be larger than the uncertainty covered by the CMC. Random contributions that cannot be known by the laboratory, such as transport uncertainties, should normally be excluded in the uncertainty statement. If, however, a laboratory anticipates that such contributions will have significant impact on the uncertainties attributed by the laboratory, the customer should be notified according to the general clauses regarding tenders and reviews of contracts in ISO/IEC 17025. • ILAC Policy for Uncertainty in Calibration (Clause 6. 5) – As the definition of CMC implies, accredited calibration laboratories shall not report a smaller uncertainty of measurement than the uncertainty of the CMC for which the laboratory is accredited. Copyright 2012 ITRI 業技術研究院

Background (6) • At it 10 th meeting, WGFF resolved “DUT uncertainty will be agreed upon by the WGFF, based on uncertainties of KC transfer standards, and included in the CMC uncertainty value. ” • The document “Proposed Definition of DUT Uncertainty” was generated on Dec. 6, 2011 and circulated among members of the WGFF Copyright 2012 ITRI 業技術研究院

Background (7) The same values of DUT uncertainties for different instruments, derived primarily from the KC reports, was proposed Measurand / Device Under Test DUT Uncertainty (k=2) Units Comments K 1 Water flow 0. 02 % Based on CCM. ff-K 1 report figures 5 to 8 and figures 19 to 22. % Based on CCM. ff-K 2 report figures 2 and 4 to 7 showing repeated cals of TS at NEL. K 2 Hydrocarbon liquid flow 0. 02 K 3 Air speed 0. 2+0. 1/u(m/s) % Based on APMP. ff-K 3 TS reproducibility, see "Air Speed" worksheet. K 5 High-pressure gas flow 0. 05 % Based on CCM. ff-K 5 b report table 2. K 6 Low-pressure gas flow 0. 05 % From Wright Flomeko 2007 paper on the uncertainty of the CCM K 6 TS. Copyright 2012 ITRI 業技術研究院

Background (8) • Feedbacks to the Dec. 6, 2011 proposal – The numbers suggested by WGFF are sensible values but the inclusion of fixed figure is not acceptable. – Development of guidance on the expression of uncertainty in CMCs and how to include repeatability, reproducibility and inter-comparison contributions to uncertainty, and uncertainty budget template. –… Copyright 2012 ITRI 業技術研究院 2018/3/15 9

Ongoing Discussion in WGFF • “WGFF Guidelines for CMC Uncertainty and Calibration Report Uncertainty” – Jun. 11, 2012 version • Read and discussed in 2012 WGFF meeting @ Colorado Springs, CO, USA • Agreed on establishment of the document and most of the content • Some revisions required and a review team formed – Jul. 24, 2012 version • Circulated among members of the review team • Comments were given – Sep. 19, 2012 version • Circulated among members of the review team • Comments being given Copyright 2012 ITRI 業技術研究院 2018/3/15 10

Introduction to the Guidelines (1) • Define Calibration and Measurement Capability uncertainty (UCMC) as the root-sum-of-squares (RSS) of: 1. the base uncertainty of the reference standard based on the GUM methodology; 2. the repeatability of calibration results measured using the best existing device. UCMC represents the 95 % confidence level uncertainty of the average of n measurements from the reference standard. The number of measurements n should match the normal procedures routinely followed during calibration of a customer’s device. Copyright 2012 ITRI 業技術研究院 2018/3/15 11

To Welch-Satterthwaite or not to W-S? “Comparison of W-S and t-Value Methods” prepared by Dr. John Wright • 4 methods were compared – Following Annex G of the GUM by using the W-S Method – Using the approach that EURAMET proposed – Using the formula – assuming that both components have infinite Do. F Copyright 2012 ITRI 業技術研究院 2018/3/15 13

To W-S or not to W-S: Recommendation “Comparison of W-S and t-Value Methods” prepared by Dr. John Wright • The Guidelines call for the use of the Welch. Satterthwaite method or the Ut 95/2 method – The W-S method follows the specific GUM recommendations – The Ut 95/2 method should be allowed since it is simple to understand the uncertainty values it produces are conservative Copyright 2012 ITRI 業技術研究院 2018/3/15 15

Introduction to the Guidelines (2) • CMC Uncertainty – Base uncertainty • uncertainty of the quantity of fluid contained within, delivered, or passed through a device under test • That each identified measurement, instrument, and influence factor is assessed for repeatability, stability, reproducibility, and historical performance is expected – Repeatability of the BED • By the nature of the test, the result will be a combination of the repeatability of the chosen DUT and that of the facility Copyright 2012 ITRI 業技術研究院 2018/3/15 16

Introduction to the Guidelines (3) • Procedure for Assessing Repeatability for a flow cal. 1. Establish the range of flows, temperatures, and other conditions listed in the scope of the facility to be covered. 2. Define a number of test flows and conditions which within the normal working practices, should demonstrate the least stable conditions and one or two which should demonstrate the most stable conditions. (The least stable results are generally found at the highest or lowest ends of the reference standard range. ) 3. Identify the BED to use for each test. Copyright 2012 ITRI 業技術研究院 2018/3/15 17

Introduction to the Guidelines (4) 4. Working within the operating procedures of the laboratory to ensure each test can proceed without interruption within a working day, perform multiple calibrations under stable flows and conditions to produce performance indicator values for the BED. The number of repeats should match the normal practices used when calibrating a customer’s device. If procedures dictate less than twenty points, the effect of the smaller sample should be fully recognised in the analysis by using the Welch-Satterthwaite or t-value approaches described below. Performance indicator may be error, meter factor, K-factor, discharge coefficient, etc. Copyright 2012 ITRI 業技術研究院 2018/3/15 18

Introduction to the Guidelines (5) 5. For each set of points, examine the data to determine the distribution to establish if a ‘normal’ or Gaussian distribution can be assumed. 6. If the distribution is approximately normal, calculate the standard deviation of the mean. If the sample size is twenty or greater, the standard deviation of the mean can be combined by RSS with the base uncertainty and expanded using a coverage factor of 2. If the sample is smaller than twenty then, 1) use the Welch-Satterthwaite method to determine the degrees of freedom and the coverage factor necessary to achieve a 95 % confidence level, or 2) multiply the standard deviation by the 95 % confidence level t-value for n-1, divide by 2 to give the standard uncertainty for RSS with other components, and use a coverage factor of 2 on the RSS to obtain the expanded uncertainty. If the distribution of the results is not normal or Gaussian, then further consideration and assessment is necessary to determine the correct statistical approach. Copyright 2012 ITRI 業技術研究院 2018/3/15 19

Introduction to the Guidelines (6) • Procedure for Assessing Repeatability for a volume cal. 1. Establish the volume range and type of volumetric equipment covered in the scope of the calibration system. 2. Select the instruments (BED) to be tested. 3. Perform n tests of each instrument at the chosen volume, where n is the number of repeated measurements normally performed on a customer’s device. 4. Work within the operating procedures of the laboratory to ensure each test can proceed without interruption within a working day. 5. Take the standard deviation of the mean of the repeated measurements and 1) use the Welch-Satterthwaite method to determine the degrees of freedom and the coverage factor necessary to achieve a 95 % confidence level , or 2) multiply the standard deviation by the 95% confidence level t-value for n-1, divide by 2 to give the standard uncertainty for RSS with other components, and use a coverage factor of 2 for the expanded uncertainty. Copyright 2012 ITRI 業技術研究院 2018/3/15 20

Note that Introduction to the Guidelines (7) • Reported Uncertainty for the Performance Indicator (UPI) – Include additional components over ubase due to: • instrumentation and characteristics associated with the DUT • fluid properties (if applicable) • repeatability or short-term reproducibility for the customer’s DUT Note: UPI must be > UCMC Copyright 2012 ITRI 業技術研究院 2018/3/15 21

Introduction to the Guidelines (8) – Associated Instrumentation and Property Uncertainties – Repeatability or Short-Term Reproducibility of the Customer’s DUT • Acceptable methods for quantifying ureprd, DUT or repeatability are (in all cases, the method must be clearly stated in the report) – The Welch-Satterthwaite method and effective degrees of freedom applied to multiple measurements at each set point. – The standard deviation of the mean of multiple measurements at each set point ( ) with statistical corrections using the t-value for the finite sample size. – The standard deviation of multiple measurements at each set point, with weighting based on the t-value at 95 % confidence for the number of points and then divided by 2 to give the standard uncertainty for RSS with other components. Copyright 2012 ITRI 業技術研究院 2018/3/15 22

Note that Introduction to the Guidelines (9) • Correlation Methods – In some reference standards, two devices can be installed in series and calibrated at the same time. This allows the application of correlation techniques or Youden analysis to separate the facility and BED repeatability. – Separating the repeatability by correlation methods is acceptable and the facility portion can be used instead of urepeat, BED to calculate UCMC. In this case, a statement in the comments section is required, such as: “Contributions to the uncertainty from the device are not included. ” Copyright 2012 ITRI 業技術研究院 2018/3/15 23