go_dtoa imprecise for last digit of 'long' values?
( disclaimer as a 'newbie': it's at least as likely that I have built in a bug somewhere, as that gnumeric or go_dtoa have one, but I have to ask to move things forward. )
in a bigger project I - mean to - have spotted a little inaccuracy of go_dtoa for long doubles when using 'shortest'.
( "!" GNM_FORMAT_g, IMHO that's what is used for cell edit display and write to files with 'conventions '-1''? )
can boil it down to a simple reproducer:
16470255126.0330839185 and
16470255126.0330839194 are distinct FP-values with 80-bit IEEE 754 'long doubles', they have a difference of
~9.313225746154785156E-10 gnumeric can calculate that but assigns the string
16470255126.033083919 to both for 'cell edit', file save with option 'shortest', or when I implement a
function using go_dtoa with option 'shortest'.
Ulf Adam's 'ryu' stays distinctive and produces
1.64702551260330839185E10 and
1.6470255126033083919E10 I think MSVC and gcc have adopted ryu for 'to_chars' ( or similar ) shortly,
but evtl. only for C++ usage?.
( it is not! a matter if I really need that accuracy, it is! a question of whether or not the building blocks of a system meet the specifications they state. the more we let go with errors the better we fulfill the IT mantra 'floating point math is inaccurate', but that's not what gnumeric set out for? For the UI it's a question to 'get what you see', editing two cells to the same string and then having a difference between them is 'irritating'. )