# Applying the manufacturer's tolerances to UUT's (and the word 'Reading') on MET/CAL

Dear members:

Perhaps you could help us on a question about UUT specifications and MET/CAL.

I'm having a difficult time explaining to my work colleagues the correct way of applying the manufacturer's tolerances to the instruments, because of the English word "Reading" and I would like to know how MET/CAL calculates the tolerance of the UUT.

Please suppose that we have a FLUKE 5520A Multifunction Calibrator that is generating 10Vdc (this is the standard value). Suppose that we want to calibrate a FLUKE 179 handheld multimeter at 10V on the 100V scale, but the multimeter is reading 6,4V.

So the error equals to 6,4V - 10V = -3,6V.

Let's suppose that the manufacturer defines the following UUT tolerance:

(1% of Reading + 0,1% of Scale)

Because of the word 'Reading' in the specifications, some colleagues say that the UUT tolerance should be calculated as:

6,4V * 1% + 100 * 0,1% = 0,164 V

But I believe that tolerance should be calculated as:

10V * 1% + 100 * 0,1% = 0,200 V

Because the word 'Reading' in the specifications really stands for "expected reading", so the specification should always apply to value that we expect to read and not the value actually read. If we apply the specification to the value actually read, we could get multiple tolerances values, and in the absurd case that the error is quite large (for example a reading of 1 V would give an error of -9V), the tolerance would be very small.

I was wondering what would be the 'correct' way of calculating the UUT specifications and how MET/CAL does it.

My sincere regards and thank you in advance,

Date
Dale Chaudiere

The "Tolerance" is entered by the procedure writer in the TOLERANCE field.   The PASS/FAIL limits, error, error as a percent of tolerance, etc. are computed by MET/CAL.   The % tolerance you refer to is used is applied one of three possible ways:

From VSET/TSET FSC help file:

 TOL_REF - SYSTEM_ACTUAL, NOMINAL, or UUT_INDICATED

If no VSET or TSET FSC is in effect, the TOL_REF values is taken the parameter "tol_ref" in the [startup] section of the MET/CAL initialization file (default name "metcal.ini").

I believe MET/CAL is shipped with a default initialization setting of System_Actual.  That is the philosophy we use at Fluke.   However it is up to your interpretation.  Fluke point of view is "% of reading" is when the device is used in the field.  When the device is being calibrated, we want the pass fail conditions based on the traceable reference value "System_Actual".

tol_ref = Nominal may sound attractive, but consider this example: 34401A 100 Ω resistance verification using the 5700A.  The 5700A is set to 100 Ω.  However, the actual traceable resistance is displayed on the 5700A front panel.  The MET/CAL 5700A driver queries the characterized resistance and returns the value in memory register MEM1.  This is why the MEMCX FSC Nominal does not contain a value.   Then when TOL_REF = SYSTEM_ACTUAL, the 34401A reading is compared to limits based on the 5700A characterized resistance.

27.004  RSLT         =
27.006  TARGET       -p
27.007  SCPI         FUNC "FRES";:FRES:RANG 100 Ohm;RES 0.0001 Ohm;NPLC 100
27.008  5700         100.0000Z                                         S  4W
27.009  TARGET       -m
27.010  SCPI         INIT;*TRG;FETCH?[I]
27.011  MEMCX  100   Z              0.010% 0.004/

Finally, consider the case of a DMM as the UUT with TOL_REF = UUT_INDICATED.  Let's say for the above test the 34401A was broken or the operator failed to connect the system properly.   Basically the PASS/FAIL limits window tracks the UUT reading, regardless of whether the reading is completely bogus.

Dale Chaudiere

The "Tolerance" is entered by the procedure writer in the TOLERANCE field.   The PASS/FAIL limits, error, error as a percent of tolerance, etc. are computed by MET/CAL.   The % tolerance you refer to is used is applied one of three possible ways:

From VSET/TSET FSC help file:

 TOL_REF - SYSTEM_ACTUAL, NOMINAL, or UUT_INDICATED

If no VSET or TSET FSC is in effect, the TOL_REF values is taken the parameter "tol_ref" in the [startup] section of the MET/CAL initialization file (default name "metcal.ini").

I believe MET/CAL is shipped with a default initialization setting of System_Actual.  That is the philosophy we use at Fluke.   However it is up to your interpretation.  Fluke point of view is "% of reading" is when the device is used in the field.  When the device is being calibrated, we want the pass fail conditions based on the traceable reference value "System_Actual".

tol_ref = Nominal may sound attractive, but consider this example: 34401A 100 Ω resistance verification using the 5700A.  The 5700A is set to 100 Ω.  However, the actual traceable resistance is displayed on the 5700A front panel.  The MET/CAL 5700A driver queries the characterized resistance and returns the value in memory register MEM1.  This is why the MEMCX FSC Nominal does not contain a value.   Then when TOL_REF = SYSTEM_ACTUAL, the 34401A reading is compared to limits based on the 5700A characterized resistance.

27.004  RSLT         =
27.006  TARGET       -p
27.007  SCPI         FUNC "FRES";:FRES:RANG 100 Ohm;RES 0.0001 Ohm;NPLC 100
27.008  5700         100.0000Z                                         S  4W
27.009  TARGET       -m
27.010  SCPI         INIT;*TRG;FETCH?[I]
27.011  MEMCX  100   Z              0.010% 0.004/

Finally, consider the case of a DMM as the UUT with TOL_REF = UUT_INDICATED.  Let's say for the above test the 34401A was broken or the operator failed to connect the system properly.   Basically the PASS/FAIL limits window tracks the UUT reading, regardless of whether the reading is completely bogus.

Marco A. Ferra

Dear Dale:

Thank you so much for your explanation and everyone else for reading. My understanding is also that, from the point of view of a calibration, the SYSTEM_ACTUAL should be the reference for calculating the UUT’s tolerances, but we had an auditing and I’m having difficulty in explaining this reasoning; the ‘standard and accepted’ approach is to calculate the tolerance from the UUT_INDICATED.

Is there an application note, some reference document or international recommendation that I show are auditors to solidify our position regarding this issue?

Best regards,