0

Setting the Exp_Unc number of Significant Digits correctly

Hi Everyone,

I've been working on getting Met/Cal to calculate measurement uncertainties using Met/Cal v7.3 sp1 and have come across a little bit of an issue.

I want to set the uncertainties to have 2 significant digits, in reading through the documentation it would seem that this is done via: VSET NSD=2

However, this doesn't work when the uncertanty calculated has leading 0's. So when the uncertainty should be 0.023, it instead becomes 0.02.

It was my understanding that the leading 0's were not supposed to be counted for purposes of significant digits.

 

Has anyone else come across this, or even better, a solution to it?

Thanks in advance!

10 comments

Date Votes
0
Avatar
Christian Tarr

Leading zeros to the right of the decimal are significant digits if they are between the decimal and number. Think about it mathematically.

0.23 = 23/100

0.023 = 23/1000

0.0230 = 230/10000 or 23/1000

 

Obviously, trailing zeros are not significant. 0.0230000000 is the same as 0.023

 

You should use "VSET NSD=3" is you want your result to be 0.023, which is 3 significant digits.

0
Avatar
David Weir-Marshall

Thanks for the reply!

What you're saying makes sense and mirrors how Met/Cal behaves.... however all the information relating to significant digits I've been able to lookup and reference, state that the preceeding 0's are not significant.

A quick google search for "significant digits" will show a number of the references I'm talking about.

 

It's that discrepency that is causing the issue for us.

0
Avatar
Christian Tarr

I stand corrected.

 

Maybe the MetCal programmers meant NSD to mean the number of digits to the right of the decimal. If your findings are repeatable, I think the solution is to include the leading zeros in your NSD setting.

0
Avatar
William (Bill) Spath

metmonkey -- I am confused here a little.  MET/CAL always reports in Scientific Notation for Expanded Uncertainty.

and the NSD does apply the normal rule for significant digits when using this mode.

Have you used a VSET to overwrite the default format?

 

0
Avatar
David Weir-Marshall

Hi Spath,

Yes, for our reporting purposes we want to see the uncertainties displayed in the same manner as the UUT displays, so we're using:

VSET UFMT = NOM

to set the uncertainties to be displayed in units of nominal.

 

Our problem is compounded as we require 2 significant digits (in a nominal format) to be reported. GIven how the uncertainties can vary depending on the UUT, we've got no way of determining what the NSD should be set to if we were going to try and compensate for the counting of 0's.

0
Avatar
Larry Newberry

MetMonkey, I had the same problem with a power meter, so here's the loop I used. The 1999 value in line 1.003 can be changed to meet any numbers of digits. 2digits=99; 3199 for 3200 count meters, etc.

  1.001  LABEL        round_digits
  1.002  MEMI         Enter value
#routine to format the output reading to 3-1/2 (significant) digits
  1.003  MATH         L[1]=1999; L[2]=0
# can't do zero or exact resolution
  1.004  JMPL         pm_format_ok      (MEM==0) || (MEM==L[1])
#
  1.005  JMPL         pm_large          MEM>=L[1]
#
  1.006  LABEL        pm_small
  1.007  MATH         MEM=MEM*10; L[2]=L[2]+1
  1.008  JMPL         pm_small          Abs(MEM)<L[1]
  1.009  MATH         MEM = (Int(MEM/10)*10)/(10^L[2])
  1.010  JMPL         pm_format_ok
#
  1.011  LABEL        pm_large
  1.012  MATH         MEM=MEM/10; L[2]=L[2]+1
  1.013  JMPL         pm_large          Abs(MEM)>L[1]
  1.014  MATH         MEM = Int(MEM)*(10^L[2])
  1.015  JMPL         pm_format_ok
#
  1.016  LABEL        pm_format_ok
  1.017  DISP         Value = [MEM]
  1.018  LABEL        exit

 

 

0
Avatar
Wang Da

Hello, my english is poor, I wish you can understand what I will say.

I met the trouble just like you and I written a function in crystaol report to deal with this problem.

First, suggest you do not set UFMT = NOM, 

because when UFMT is NOM, the uncertainties to be displayed in units of nominal.

for example, the measured value is 9.997mV, the uncertainty is 0.013mV, when the units if nominal,

the uncertainty is 0.0000013mV, both NSD=2 or NSD=3, the uncertainty value in database will be stored as

 0.0 V or 0.00 V. I feel it's a bug of METCAL 7.3.

so, we can not use UFMT=NOM,we should use Scientific notation, then the result is 1.3E-5V.

Then in Crystal report, we translate 1.3E-5V to 0.0013mV. 

I can email the function code of Crystal report to you  when I back to my office. Please tell me your email address

0
Avatar
Wang Da

Hello, my english is poor, I wish you can understand what I will say.

I met the trouble just like you and I written a function in crystaol report to deal with this problem.

First, suggest you do not set UFMT = NOM, 

because when UFMT is NOM, the uncertainties to be displayed in units of nominal.

for example, the measured value is 9.997mV, the uncertainty is 0.013mV, when the units if nominal,

the uncertainty is 0.0000013mV, both NSD=2 or NSD=3, the uncertainty value in database will be stored as

 0.0 V or 0.00 V. I feel it's a bug of METCAL 7.3.

so, we can not use UFMT=NOM,we should use Scientific notation, then the result is 1.3E-5V.

Then in Crystal report, we translate 1.3E-5V to 0.0013mV. 

I can email the function code of Crystal report to you  when I back to my office. Please tell me your email address

0
Avatar
Wang Da

I written a function to translate Scientific notation to nominal expression.

================================

 

//本函数主要修正不确定度2.0E-005表达中的“.0”部分
local numbervar e_pos;
local numbervar exp_len;
//字符型表示的不确定度的位数长度,如果没有,则为0
local numbervar num_exp;//不确定度E前面部分,数值型
local StringVar str_exp;
local StringVar l_exp;
//不确定度表示E前面部分,例如1.0E-004,就提取1.0。如果最后2位是“.0”,则去掉“.0”
local StringVar r_exp;
//不确定度表示E前面部分,例如1.0E-004,提取004
local StringVar cor_exp;
//修正后的不确定度表示
local numbervar exp_ind;
//科学技术法表示的指数部分,1.0E-004就是-4
local numbervar poin_len;
//从测量值判断小数位数
local stringvar str_varq;
//测量值
local numbervar vaqq_ascii;
str_exp :={Ver_7_Certificates.exp_uncert};
str_varq:={Ver_7_Certificates.varq};
poin_len:=0;
If Instr(str_varq,".") > 0 Then
    poin_len:=length(str_varq)-Instr(str_varq,".")
Else
    poin_len:=0;
// 如果从第2位开始的2位是“.0”,就把".0"换成空
IF Length(str_exp)>4 Then //长度大于4,则是用科学计数法表示,进行单位换算
    (
    IF Mid(str_exp,2,2)=".0" Then 
        (cor_exp:=replace(str_exp,".0","");l_exp:=left(cor_exp,1))
    ELSE
        (cor_exp:=str_exp;l_exp:=left(cor_exp,3););
//准备单位换算,根据测量值单位u,m,k,M,换算系数为10的-6,-3,3和6次方
    r_exp:=right(cor_exp,4); 
    r_exp:=replace(r_exp,"+","");  
    r_exp:=trim(r_exp);
//发现case 不区分字母大小写,用ASCII代替
    IF length(trim({Ver_7_Certificates.varq_p}))>0 then 
        vaqq_ascii:=AscW(trim({Ver_7_Certificates.varq_p}))
    ELSE
        vaqq_ascii:=-1;
    if numerictext(r_exp) then
        (
        exp_ind:=tonumber(r_exp); 
        //Select {Ver_7_Certificates.varq_p} 
        //Case "M": exp_ind:=exp_ind-6
        //Case "p": exp_ind:=exp_ind+9
        //Case "u": exp_ind:=exp_ind+6
        //Case "m": exp_ind:=exp_ind+3
        //Case "k": exp_ind:=exp_ind-3
        Select vaqq_ascii
        Case 112: exp_ind:=exp_ind+9    //p
        Case 117: exp_ind:=exp_ind+6    //u
        Case 109: exp_ind:=exp_ind+3    //m
        Case 107: exp_ind:=exp_ind-3    //k
        Case 77:  exp_ind:=exp_ind-6    //M
         Default:  exp_ind:=exp_ind+0;
        num_exp:=tonumber(l_exp);
        num_exp:=num_exp*(10^exp_ind);
        cor_exp:=totext(num_exp,poin_len);
        IF left(cor_exp,1)="." Then cor_exp:="0"+cor_exp;
        //如果最后是0,则删除,比如0.00150
        While Right(cor_exp,1)="0" DO
            (
            exp_len:=length(cor_exp);
            cor_exp:=Left(cor_exp,exp_len-1);
            );
        cor_exp     
    )
    else
        "error"
    )

0
Avatar
David Weir-Marshall

@Rware: Thanks for your reply. The issue with the uncertainties doesn't seem to be linked to the output format of the unit, but I appreciate your help!

 

@Dewbow: Thanks for your help! We came to the same conclusion of this being a bug in 7.3, but we figured that since there is a newer version out, this bug will never get fixed. What you've stated is exactly our issue.

I was hoping to avoid having to make Crystal Reports convert scientific notation, but from what you've said it seems like that's going to be the only real way of dealing with this.

Thanks for the code sample, it'll save me quite a bit of time! Owe you one!

Please sign in to leave a comment.