**Chad D.**

Hi Samuel313, your procedure appears to be storing each value used by MEMCX in memory registers. I assume that your MATH FSC statements for M1 and M2 are actually using the numeric memory register syntax: M[1] = 0.07 and M[2] = 0.07. If not, then these lines are creating local named variables M1 and M2, and not actually populating memory registers M[1] and M[2]. This may explain part of the problem? Here is the syntax to use if you want these registers to be compatible with the tolerance field of your MEMCX:

1.005 MATH M[1] = 0.07; M[2] = 0.07

The nominal value of 0.000 is stored in MEM1, which MET/CAL stores as the unformatted number 0. MET/CAL attempts to figure out the UUT resolution based on the information provided. I think these conditions are the ones that apply to your scenario, as described in the VSET/TSET FSC help file, but I suspect that 2.d is failing since the tolerance values are being taken from a register, leaving condition 2.e to set the resolution. Since MEM1 contains your nominal value as 0, your resolution is also set to 0 instead of 0.000:

*Unless overridden, MET/CAL attempts to infer the UUT's resolution based on information in the procedure.*

*2. If the test evaluation step is a MEMC or MEMCX statement:*

*d. Otherwise, if the MEMC or MEMCX TOLERANCE field specifies a tolerance in absolute units ('U'), and there are one or more digits to the right of the decimal point, the UUT resolution is based on the number of digits to the right of the decimal point in the tolerance field specification.*

*e. Otherwise, an attempt is made to guess the resolution of the UUT based on the NOMINAL value. The algorithm involves formatting the NOMINAL value with up to 10 significant digits, and then counting the number of digits to the right of the decimal point. In this case the NOMINAL value is determined as follows:*

You may be able to address the resolution issue with the MATH FSC's FMT() function. You can also use TSET or VSET UUT_RES = 0.001 to define UUT resolution. Another method would be to define your system accuracy using an ACC FSC, and set its nominal field to 0.000dB. However, if your expected nominal value is always 0.000dB, then the most direct route would be to insert 0.000dB is your nominal value in the MEMCX.

1.010 MEMCX 0.000dB -M1U +M2U

I am not sure what the MEM = 119.934 dBµV statement is intended to do, but please know that the contents of MEM will be replaced by the 34401 reading in step 1.031.

I hope this helps!

-Chad D.

## 9 comments