Problem with statistical routines (MeanAndStdDev and
MomentSkewKurtosis)
lazarus / 62-General

----------------------------------------------------------------------
Чебурашка · 2022-02-03 13:56

Hello,
I was playing with the statistical routines in Math and I came across
a behaviour that I am not sure is ok.

The portion of code "All samples with distinct values" fails only for
0 and 1.
This is senseful because for the requested calculations at least 2
samples are requested.
Maybe putting a check in the method could slow down excessively on
frequent repeated calls .

The portion of code "All samples with same value" fails periodically
(btw in correspondence of powers of 2).
This probably due to the fact that the samples are a fake distribution
with no spreadness.

Is that the caller obeyed to make a preverification that the sample
set is somehow spread to a minimum?


Code: Pascal  [Select][+][-]window.onload = function(){var x1 =
document.getElementById("main_content_section"); if (x1) { var x =
document.getElementsByClassName("geshi");for (var i = 0; i program
Project1;uses  SysUtils, Math;var  a: array of float;  Mean, StdDev:
float;  m1, m2, m3, m4, Skewness, Kurtosis: float;  n, i:
Integer;begin  // All samples with same value  for n := 0 to 99 do   
try      SetLength(a, n);      for i := 0 to n - 1 do a[i] := 9;     
MeanAndStdDev(a, Mean, StdDev);      MomentSkewKurtosis(a, m1, m2, m3,
m4, Skewness, Kurtosis);      Writeln(Format('%d done', [n]));   
except      Writeln(Format('%d failed', [n]));    end;   // All
samples with distinct values  for n := 0 to 99 do    try     
SetLength(a, n);      for i := 0 to n - 1 do a[i] := 9 + i;     
MeanAndStdDev(a, Mean, StdDev);      MomentSkewKurtosis(a, m1, m2, m3,
m4, Skewness, Kurtosis);      Writeln(Format('%d done', [n]));   
except      Writeln(Format('%d failed', [n]));    end; end.

----------------------------------------------------------------------
Чебурашка · 2022-02-07 11:11

Quote from: tt on February 03, 2022, 02:56:17 pmIs that the caller
obeyed to make a preverification that the sample set is somehow spread
to a minimum?

Hi, does anybody have a clue on the question posed?
Thank you vm

----------------------------------------------------------------------
wp · 2022-02-07 11:39

Look at the definition of skewness and kurtosis in wikipedia: https://
en.wikipedia.org/wiki/Skewness#Fisher's_moment_coefficient_of_skewness
and https://en.wikipedia.org/wiki/Kurtosis#Pearson_moments: All of
them divide some number by the standard deviation of the data set. But
when all values are equal the standard deviation is zero, and this
results in the error observed.

The second part of your demo ("All samples with distinct values")
crashes for the same reason because you start your n loop with 0. With
n=0 your data array is empty - having no data, the demo crashes
already in the calculation of the mean. With n=1, the standard
deviation cannot be calculated since it requires a division by n-1
(=0) --> crash. Only when the loop starts with n=2 the program runs
successfully.

----------------------------------------------------------------------
Чебурашка · 2022-02-07 12:44

Ok.

So I assume that before calling MomentSkewKurtosis() it is necessary
to make sure (StdDev > 0).