On Tue, Aug 4, 2009 at 9:26 PM, Ross McFarland<email address hidden> wrote:
> On Tue, Aug 4, 2009 at 7:50 PM, Ross McFarland<email address hidden> wrote:
it was passing in width (the size of the buffer, 341) for ndigits
which is supposed to be the (max) number of digits to include in the
string (buffer should be this size or larger.) when 341 is passed it
tries to convert to a string that is higher precision than the double
it's copying from can reliably be used for (> DBL_DIG digits) and thus
gets noise. this just happens to work on linux since it gives out the
double value of exactly 2 in the first place.
good news is that fixes the log(3, 9) func_math test. bad news is that
others are now failing. looking at the failures i'm not 100% sure yet,
but i think the expected values may be wrong. digging yet more...
On Tue, Aug 4, 2009 at 9:26 PM, Ross McFarland<email address hidden> wrote:
> On Tue, Aug 4, 2009 at 7:50 PM, Ross McFarland<email address hidden> wrote:
> problem is in drizzle mystrings/dtoa.cc, dtoa
found it:
res= dtoa(x, 4, type == MY_GCVT_ARG_DOUBLE ?
min( width, DBL_DIG) : width,
&decpt, &sign, &end);
should be:
res= dtoa(x, 4, type == MY_GCVT_ARG_DOUBLE ?
min( width, DBL_DIG) : min(width, FLT_DIG),
&decpt, &sign, &end);
it was passing in width (the size of the buffer, 341) for ndigits
which is supposed to be the (max) number of digits to include in the
string (buffer should be this size or larger.) when 341 is passed it
tries to convert to a string that is higher precision than the double
it's copying from can reliably be used for (> DBL_DIG digits) and thus
gets noise. this just happens to work on linux since it gives out the
double value of exactly 2 in the first place.
good news is that fixes the log(3, 9) func_math test. bad news is that
others are now failing. looking at the failures i'm not 100% sure yet,
but i think the expected values may be wrong. digging yet more...
--
-rm