Computer Science Canada

floating point precision
Author:  btiffin [ Wed Apr 09, 2008 1:20 pm ]
Post subject:  floating point precision
Hello

Does anyone know of any floating point implementations that get round this problem?

code:
Ruby
irb(main):001:0> 10.3 - 10
=> 0.300000000000001

python
>>> 10.3 - 10
0.30000000000000071

REBOL 2
>> 10.3 - 10
== 0.300000000000001

REBOL 3 Alpha
>> 10.3 - 10                                                                   
== 0.3000000000000007
>> $10.3 - 10                                                                   
== $0.3


R3 has a highly precise money! datatype, but doing math with money is just not computer science "floating point arthimetic"

Just curious if anything new is out there; we'd like to build R3 with the best fp implementation that we can. It may well be that ieee 754 and 854 are the best we have access to, but I thought I'd ask. I'd be interested if anyone knows how Walter Bright did it in D. And just so ya know, this is my first stop before I RTFM; but I find it nicer to start with Ask A Friendly Human Smile

Cheers
Author:  Nick [ Wed Apr 09, 2008 2:35 pm ]
Post subject:  RE:floating point precision
is there ways to cast floats into ints in those languages (honestly don't know) if so,

number * 10^decimals wanted

cast into int

nnumber / 10 ^ decimals wanted

so...

0.33333 * 10 ^ 2 = 33.333

cast = 33

33 / 10^2 = 0.33

and voila
Author:  OneOffDriveByPoster [ Wed Apr 09, 2008 3:17 pm ]
Post subject:  Re: floating point precision
Best that I know of:
http://www2.hursley.ibm.com/decimal/decnumber.html
Author:  Dan [ Wed Apr 09, 2008 3:21 pm ]
Post subject:  RE:floating point precision
If i remember right this might be more of a problem on the cpu side rather then the language.

How to get around the problem realy dependens on what you are using the floating points for. If you are dealing with money it is not noramly repsented to anything past a cent and is rounded to the nearest cent.

I am not realy sure what R3 is so this might be of no help.
Author:  md [ Wed Apr 09, 2008 3:35 pm ]
Post subject:  RE:floating point precision
using the IEEE formats is the way to go - they are VERY well documented and hardware support for floating point is a requirement for every language these days.
Author:  wtd [ Wed Apr 09, 2008 5:17 pm ]
Post subject:  RE:floating point precision
Your "10.3" value was never really "10.3" as far as the computer was concerned, so the answer is completely accurate.

Having an accurate money type is fantastic, but everyone else solved the problem in a much smarter way: they side-stepped it. Instead of calculating in dollars they calculate everything in pennies, and then just format the output for dollar-obsessed end-users. Smile
Author:  btiffin [ Wed Apr 09, 2008 6:43 pm ]
Post subject:  Re: floating point precision
Awesome feedback gentlemen, I knew I'd be talking to the right crowd.

OneOffDriveByPoster; Great resource. Looks worthy of analysis.

In REBOL Release 3, money! is not treated as 64bit ieee, it's a decimal encoding, not BCD exactly, but something developed by a Math teacher. Ladislav Mecir. A true Computer Scientist.

money! stays accurate to 26 digits. R3 is still alpha, so there may be the odd edge-case, but Ladislav knows his bits. Smile

>> $123456789012345678901234.56 + $0.01
== $123456789012345678901234.57

Even Bill would foam at the mouth given that much money! But a COBOL programmer might foam seeing

>> $10.3333 - 10
== $0.3333

As his code slowly accumulated all the fractional pennies into his off shore account. Smile

But anyway, now I'm digressing. So no new ieee trends it seems? And OneOffDriveByPoster, it'll take a while to peruse that doc. Thanks.

And Walter uses the 80bit format in his code libraries, so the rounding looks much better in D than in C (for the 10.3 case). They will just have different approximation edge-cases Smile

By the way; R3 is about to unleash a self reflective test suite. With the 56 data types, it could generate about 1 million test vectors. Any volunteers for eyeball analysis of a million test results? You'll be able to count yourself in a small team of rebols. Smile

Cheers
Author:  OneOffDriveByPoster [ Wed Apr 09, 2008 7:02 pm ]
Post subject:  Re: floating point precision
"754r" is the current IEEE trend; it is very relevant to the link that I provided.
Author:  btiffin [ Wed Apr 09, 2008 9:22 pm ]
Post subject:  Re: floating point precision
OneOffDriveByPoster @ Wed Apr 09, 2008 7:02 pm wrote:
"754r" is the current IEEE trend; it is very relevant to the link that I provided.


Yep, thanks much. It's a good read that, just starting. 745r it is. And I'll apologize in advance, twice I had to retype your nick, from OffByOneDriveByPoster. So if you ever see that slip through, please know that I mean well. It's a habit of mine, to be off by one. Smile

From what I can tell so far, (and I haven't studied the bit twiddling since my first hardware FPU), 64 bit and 80 bit intermediates will still have the same edge-cases for approximations. (As far as I can suss out with a quick grok, but I have a lot more RTFM in front of me) But, 128bit floats. Cool. My old Radio Shack TRS/80 would have loved grinding through those. With about 8K usable, it could have held a whopping 500 numbers, less a couple for code. Smile The trip to Mars?
code:
10 FOR I = 1 TO 500  LAUNCH


Cheers
Author:  OneOffDriveByPoster [ Wed Apr 09, 2008 9:38 pm ]
Post subject:  Re: floating point precision
btiffin @ Wed Apr 09, 2008 9:22 pm wrote:
And I'll apologize in advance, twice I had to retype your nick, from OffByOneDriveByPoster.
"One off" is my way of referring to "off by one". That is what it means--an excuse for me to be off-by-one once in a while at compsci.ca Smile.

:


You can syndicate this boards posts using the file backend.php or view the topic map using sitemap.php.

Terms of Use | Privacy Policy