Rounding data in report

 Hi All!

I have two issue with data indicating in MET/CAL

1) After procedure complete I see that data in report rounded to one digit after dot.

For example: 

in test result window under debug of precedure I see that UUT indicated is 32.99925mV value.

When I execute same procedure in RunTime I see in the report that UUT indicate value is 33.0mV.

2) How to ascess to result file (which I see during Debug the procedure)  after procedure performing.


Date Votes
Michael Johnston

For reported resolution, the following is the heirarchy MET/CAL uses to determine what resolution is reported in the report:

  1. An instrument FSC used as an EVAL with a value in nominal field (5520 FSC with a TOL field, for example)
  2. A MEMC or MEMCX FSC with a value in the nominal field
  3. An ACC FSC with a value in the nominal field
  4. An instrument FSC used as a setup with a value in the nominal field
  5. A MEMC or MEMCX FSC with a tolerance component in terms of units (0.05U would represent a resolution of 0.01)
  6. TSET/VSET UUT_RES setting

Depending on your code, you're probably using a register to fill in a nominal in an instrument or MEMC/MEMCX FSC, so it's pulling your resolution from one of the later sources and it's not set up the way you're wanting. Was this a procedure you wrote or got from Fluke? Without seeing the actual code, it's hard to say exactly what needs to be changes for it to display properly for you.

You can't access the results from a debug run in Editor, at least as far as I'm aware, though someone can correct me if that's wrong. It's necessary to run the procedure in Run Time to see the final display on the report. But, you should be able to ensure it turns out right by cleaning up your code to watch for the heirarchy I mentioned above.

Dmiti Kubarev

 Thank You for response, Michael!

I try to write my own procedure (see attached screenshot).

So, now I see that in sertificate displays right value from MEMCX, but from  statements 3458 value also displays.
How modoficate procedure for 3458 statement nothing displays in the test result file,
(screenshot of test result window and final certificate attached)
Attachment not imported: code.jpg
Michael Johnston

Right now your 3458 FSC is actually acting in place of your MEMC/MEMCX in terms of evaluating the test, so just delete lines 2.001 and 2.002 and you'll be fine. Add zeros after the decimal of the nominal field of the 3458 like you did in the MEMCX one and that will handle the resolution aspect.

Nathan Ryder

I would also strongly suggest if you are using MEMCX and MEMC that if your UUT is sourcing into a reference (DMM) then use MEMC not MEMCX 

And if its a measuring UUT (DMM) then use MEMCX.....

It will ensure your results are correct as to whats the reference and whats the UUT.

And I personally handle the resolution of the UUT on the MEMCX/MEMC line so below it will report it as having 3 decimal places.

See attached pic...


Attachment not imported: memc_memcx.jpg

Please sign in to leave a comment.