0

Resolution problem

Dear All,

I generate a procedure for the power supply with a resolution of 0.1V with accuracy of 1% FS which is 0.3V at 30V FS.

My problem is why the resolution of lower and upper limit is become 0.01 instead of 0.1 on the calibration result. See below result:

Nominal Value          Lower Limit                  Measured Value                 Upper Limit

   2.0V                          1.70                                 2.1                                           2.30 

Here is my generated proc.

1.001       ASK-    R                   P                F                     W

1.002      ASK+    K

1.009      DISP    Set the UUT to 2V

1.010      MEMI    Enter the measured reading

1.011      MEMC   2.0V                   0.3U

Any suggestion..

Thanks

Vher

 

4 comments

Date Votes
0
Avatar
Michael Johnston

At some point before the MEMC line, use the following:

TSET UUT_RES = 0.1

 

That should fix it. Just as a note, the UUT_RES value is in base units, so if your nominal on your MEMC is mV and the resolution is 0.1 mV, you'd need to use UUT_RES = 0.0001.

0
Avatar
Oliver Vher

Dear dmjohnston,

I already put the TSET UUT_RES = 0.1 before the MEMC line but still the same with my concern, the resolution of the lower and upper limit become 0.01

Regards,

Vher

 

0
Avatar
Dale Chaudiere

System Actual (SA) is always logged in the result with one more digit of resolution than UUT Indicated (UI).  MEMC is typically used when a measuring reference (e.g. 8508A) is used to verify a sourcing UUT (e.g. 4-20 mA Process Calibrator output).  If this test is a source reference verifying a measuring UUT, than you should use MEMCX.  Remember, SA is the value established by the reference device.  This can be confusing when a measurement device is calibrating a source.

Example:  5500A DC Voltage verification using a 8508A.

System Actual (SA):  8508A reading

UUT Indicated (UI):  5500A programmed/set voltage

0
Avatar
Oliver Vher

Hi Dale,

Thanks..yes your correct when I change the MEMC to MEMCX FSC the nominal value resolution of the UUT is same now. It is sufficient to use MEMCX evaluation but depending on the type of test.

Thanks and regards,

Vher

Please sign in to leave a comment.