|
|
Post by timetraveller on Oct 20, 2022 2:05:26 GMT
As far as I can see, it seems like the 64-bit variables only have 32-bit precision. All the operations also only seem to give results in 32-bit precision. Why is this? Is it planned to be fixed in future versions? For example if you try the following:
a## = 1.111222333444555666777888999
b## = 1.111222333444555
c## = a## - b##
PRINT c##
the result should be 6.66777888999 E-16 if you really have 64-bit precision, but for me I don't get that result. |
|
|
|
Post by bplus on Oct 20, 2022 3:04:39 GMT
Hello again timetraveller, Floating point math sucks Dim As _Unsigned _Integer64 a, b, c
a = 1111222333444555666777888999
b = 1111222333444555000000000000
c = a - b
Print c |
|
jack
New Member
Posts: 9 
|
Post by jack on Oct 21, 2022 13:57:37 GMT
timetraveller you need to use the ## suffix on the literals
a## = 1.111222333444555666777888999##
b## = 1.111222333444555##
c## = a## - b##
Print c##
C gives 6.667843360785852269e-16 remember that QB64 converts a _Float to double for printing or Str$
P.S. you are subtracting 2 numbers that are almost the same, _Float holds about 19 digits, take away 16 digits and you are left with only 3 meaningful digits
|
|
