# Tolerance goes to zero for nominal values of zero

I am measuring the frequency response of a microphone preamplifier by injecting a signal into it with a SRS DS360 signal generator and measuring the voltage with an Agilent 34401A DMM. The signal is nominally 120 dBµV and referenced from 1kHz. The signal generator's amplitude and frequency, expected nominal value, and tolerances are read from a text file for each test point. Some of the frequencies in question should not differ from the 1 kHz level measured by the DMM, so the nominal difference is given to be 0. An example is as follows.

@freq = 5011.9 Hz
@amp = 1 Vrms
@kHz_ref = 119.976 dBµV
@dB_ref = 1E-6
M1 = 0.07 dBµV
M2 = 0.07 dBµV
MEM = 119.934 dBµV
MEM1 = 0.000 dBµV

1.030  PORT         [@DS360]FREQ [V @freq]; AMPL [V @amp]VR
1.031  PORT         [@34401]READ?[I]
1.032  MATH         MEM = 20*log(MEM/@dB_ref) - @kHz_ref
1.033  TSET         TDESC = [V @freq] Hz
1.034  MEMCX        dB             -M1U +M2U
2.001  END

Whenever the nominal value is 0.000, the tolerances reported also collapse to zero, even though the tolerances are expressed in units and not percentages. Is this because MET/CAL converts tolerances to a ratio of the nominal value or a function of it expecting a nonzero value for the expected result?

Also, the expanded uncertainty is always reported much higher when the nominal value is zero. Even if the value is given as 0.000, MET/CAL stores the value as 0, so the UUT resolution is significantly larger than other test points. I could set UUT_RES to default to three decimal places, but I'd prefer a more elegant solution.

### 9 comments

Date
Chad D.

Hi Samuel313, your procedure appears to be storing each value used by MEMCX in memory registers. I assume that your MATH FSC statements for M1 and M2 are actually using the numeric memory register syntax: M[1] = 0.07 and M[2] = 0.07. If not, then these lines are creating local named variables M1 and M2, and not actually populating memory registers M[1] and M[2]. This may explain part of the problem? Here is the syntax to use if you want these registers to be compatible with the tolerance field of your MEMCX:

` 1.005 MATH M[1] = 0.07; M[2] = 0.07`

The nominal value of 0.000 is stored in MEM1, which MET/CAL stores as the unformatted number 0. MET/CAL attempts to figure out the UUT resolution based on the information provided. I think these conditions are the ones that apply to your scenario, as described in the VSET/TSET FSC help file, but I suspect that 2.d is failing since the tolerance values are being taken from a register, leaving condition 2.e to set the resolution. Since MEM1 contains your nominal value as 0, your resolution is also set to 0 instead of 0.000:

Unless overridden, MET/CAL attempts to infer the UUT's resolution based on information in the procedure.

2. If the test evaluation step is a MEMC or MEMCX statement:

d. Otherwise, if the MEMC or MEMCX TOLERANCE field specifies a tolerance in absolute units ('U'), and there are one or more digits to the right of the decimal point, the UUT resolution is based on the number of digits to the right of the decimal point in the tolerance field specification.

e. Otherwise, an attempt is made to guess the resolution of the UUT based on the NOMINAL value. The algorithm involves formatting the NOMINAL value with up to 10 significant digits, and then counting the number of digits to the right of the decimal point. In this case the NOMINAL value is determined as follows:

You may be able to address the resolution issue with the MATH FSC's FMT() function. You can also use TSET or VSET UUT_RES = 0.001 to define UUT resolution. Another method would be to define your system accuracy using an ACC FSC, and set its nominal field to 0.000dB. However, if your expected nominal value is always 0.000dB, then the most direct route would be to insert 0.000dB is your nominal value in the MEMCX.

` 1.010 MEMCX 0.000dB -M1U +M2U`

I am not sure what the MEM = 119.934 dBµV statement is intended to do, but please know that the contents of MEM will be replaced by the 34401 reading in step 1.031.

I hope this helps!

-Chad D.

Samuel Anderson

Chad,

For brevity I listed M[1] and M[2] as stated above in my original post, that is not the issue. My concern is not really about the resolution, I can fix that, through the methods you listed above. I just find it unfortunate that the values as read from my text file are not stored in MET/CAL as written. My real issue is that the tolerance values stored in M[1] and M[2] are written to the results as 0 if the nominal is also 0. I have included the uncertainty report from 10 runs in file "prm831_sweep_10081.xls" and the file "prm831_freq.xls" contains the values read by the procedure for each test point, and contains the amplitude, frequency, nominal, tolerances, and a placeholder for standard deviation. I have also included an image of the results of the procedure as run in the runtime. You will see that the tolerance collapses to 0 whenever the expected value is 0. The entirety of the procedure used can be found below.

INSTRUMENT:            Sub Frequency_Sweep
DATE:                  2013-07-05 11:29:53
AUTHOR:                Sam Anderson
REVISION:              \$Revision: 1\$
ADJUSTMENT THRESHOLD:  70%
NUMBER OF TESTS:       2
NUMBER OF LINES:       52
CONFIGURATION:         HP 34401A
CONFIGURATION:         SRS DS360
=============================================================================
STEP    FSC    RANGE NOMINAL        TOLERANCE     MOD1        MOD2  3  4 CON
1.001  MATH         str = READ(@file,@counter)
1.002  MATH         @amp = NUMN(str,1)
1.003  MATH         @freq = NUMN(str,2)
1.004  MATH         MEM1 = NUMN(str,3)
1.005  MATH         M[1] = MEM1 - NUMN(str,4)
1.006  MATH         M[2] = NUMN(str,5) - MEM1
1.007  MATH         st_dev = NUMN(str,6)

#Perform Tests
1.008  IF           @freq > 1000
1.009  PORT         [@34401]SENS:DET:BAND 200
1.010  ELSEIF       @freq > 200
1.011  PORT         [@34401]SENS:DET:BAND 20
1.012  ELSE
1.013  PORT         [@34401]SENS:DET:BAND 3
1.014  ENDIF
1.015  MATH         gen_Vpp = @amp*2*sqrt(2)
1.016  MATH         DMM_Vpp = 0.5
1.017  MATH         acc1 = ACCV2("SRS DS360","Ampl",gen_Vpp,@freq)
1.018  IF           @freq < 3 && @freq > 2
1.019  MATH         acc2 = 0.00223
1.020  ELSEIF       @freq <=2
1.021  HEAD         Frequency too low for Multimeter
1.022  END
1.023  ELSE
1.024  MATH         acc2 = ACCV2("HP 34401A","Volts",DMM_Vpp,@freq)
1.025  ENDIF
1.026  MATH         sys_acc_V = RSS2(acc1,acc2)
1.027  MATH         sys_acc = 10*log((gen_Vpp+sys_acc_V)^2/gen_Vpp^2)
1.028  TSET         U1 = [V sys_acc]
1.029  TARGET       -m
1.030  PORT         [@DS360]FREQ [V @freq]; AMPL [V @amp]VR
# 1.031  WAIT         [D500]
1.031  TARGET       -m
1.032  PORT         [@34401]READ?[I]
1.033  MATH         MEM = 20*log(MEM/@dB_ref) - @kHz_ref
1.034  TSET         TDESC = [V @freq] Hz
1.035  MEMCX        dB             -M1U +M2U
2.001  END

Attachment not imported: tolerance_issue.png
Chad D.

The incorrect UUT resolution seems to be the root of all of the problems described.  MET/CAL is essentially trying to guess the UUT resolution from the information available (which it normally does a very good job of), but it is unable to guess correctly since all values are being taken from registers.  Your procedure will need to provide MET/CAL with more information to properly define UUT resolution using any of the methods described above.

-Chad

Samuel Anderson

As I've looked into the issue further, it may be an issue in MET/TEAM, not MET/CAL per se. During the runtime, the results are referenced to the tolerances given and the appropriate pass/fail condition is given. The %tol and error are also appropriate as shown in the attached image. In the report however, the values for the upper and lower tolerance are 0 for nominal values of 0. I can't explain why this happens.

Attachment not imported: samplereport.pdf
Chad D.

Thanks for the detailed explanations of everything!  I will share this link with our support group.

-Chad D.

Samuel Anderson

Chad,

Any word from the Support group? I haven't heard anything yet.

Chad D.

Hi Samuel313, nothing yet but many of us are just starting to return from NCSLi in Nashville.  I will follow up with them.

-Chad

Samuel Anderson

Chad,

I have been working on the procedure and changed how I had the ASCII text files written. I can't be sure what changes made the difference, but I made the values tab-delimited, and added trailing zeros to my data for x.xx precision on all values. Together with setting UUT_RES = 0.001 it must have fixed the problem.

Attachment not imported: ascii_file.png
Chad D.

Hi Samuel, I'm glad to hear the problem is fixed!

-Chad

Please sign in to leave a comment.