# MEMC / MEMCX Fixed Decimal Places Needed

Greetings.

Fairly new to MET/CAL and first time posting.

I have been having trouble with getting decimal places in my results using MEMC / MEMCX.

What I understand is happening the numeric register is parsing INT when I try to assign 4.00 to it when I need it to be a fixed-point. So in all my results I have been seeing integers.

I have not been able to see any: Test Tolerance, System Accuracy, T.U.R, Uncertainty.

Below is an extract from my procedure. I need in my results to display 2 decimal places when I do MEMC. Is there a way to have the nominal field as a fixed-point variable?

# --- CLASS SELECTION ---
2.001 TARGET -p
2.002 LABEL CLASS
2.003 MEMI SELECT TORQUE WRENCH CLASS: 1 = A,B,G 2 = D,E
2.004 IF (MEM == 1)
2.005 TOL 4.00%
2.006 ELSEIF (MEM == 2)
2.007 TOL 6.00%
2.008 ELSE
2.009 JMPL CLASS
2.010 ENDIF

# --- MEASUREMENT POINTS ---
2.011 TARGET -p
2.012 LABEL MEASURE
2.013 MEMI SELECT 20% Measurement Point
2.014 MATH M[1] = MEM
2.015 MEMI SELECT 60% Measurement Point
2.016 MATH M[2] = MEM
2.017 MEMI SELECT 100% Measurement Point
2.018 MATH M[3] = MEM
2.019 OPBR Proceed to next step = YES
2.019 OPBR Repeat Input Measurement Points = NO
2.020 IF (MEM1 == -1)
2.021 JMPL MEASURE
2.022 ENDIF

# --- UNIT SELECTION ---
2.023 TARGET -p
2.024 LABEL UNITS
2.025 MEMI SELECT TORQUE WRENCH UNITS: 1 = Nm 2 = lbf
2.026 IF (MEM == 1)
2.027 JMPL NM
2.028 ELSEIF (MEM == 2)
2.029 JMPL LBF
2.030 ELSE
2.031 JMPL UNITS
2.032 ENDIF

# --- NEWTON METERS ---
2.033 LABEL NM
2.034 RSLT =
2.036 TARGET -p
2.037 RSLT =
2.039 HEAD {20% of Full Scale}
2.040 MATH MEM1 = M[1]
2.041 MEMI Input the 1st Measurement
2.043 MEMC Nm TOL
3.001 MEMI Input the 2nd Measurement
3.002 MEMC Nm TOL
4.001 MEMI Input the 3rd Measurement
4.002 MEMC Nm TOL
5.001 MEMI Input the 4th Measurement
5.002 MEMC Nm TOL

Date

## Official comment

Michael J.

Hello David,

The VSET/TSET section has a pretty good explanation of how MET/CAL attempts to determine your DUT resolution automatically in the UUT_RES parameter description.

Essentially, there is a hierarchy of locations it looks for a value to use, both for reporting the resolution, and for using it in the uncertainty calculations.

1. Nominal Field of a legacy instrument FSC acting as an evaluation.
2. Nominal Field of a MEMC/MEMCX FSC.
3. Nominal Field of an ACC FSC.
4. Tolerance Field of a MEMC/MEMCX FSC where a units-based tolerance is included (the resolution of the U component is used).

This hierarchy can be overridden using the VSET/TSET UUT_RES parameter. Whenever it is used, MET/CAL will ignore the other locations and use the value of the UUT_RES setting.

One important thing to note is that unlike the U component of a tolerance field, UUT_RES is in terms of base units, not the reported units. So if you are measuring 100 mV and you put 0.1U in the tolerance field, you get 0.1 mV, but UUT_RES = 0.1 would be 0.1 V, or 100 mV.

The easiest solution for your case, since you are using blank nominal fields, would be to use VSET UUT_RES to set the resolution across the procedure.

I'll note that you should also add an ACC statement in your code to allow MET/CAL to calculate your uncertainties properly.