Precision

Doomulation · Oct 11, 2005

So, can you at least help out a lil' with precision here? The precision is too bad using doubles (since 42 / 10 stored in doubles results in 4.20000000000002 or something). SO, I need a way to get around it. An own precision mechanism perhaps?

I need the precision because I was writing a little calculator for fun's sake, and for those, precision is a good thing.

zenogais · Oct 11, 2005

For higher precision floating point values you might want to investigate scaled real values. I have an implementation written as a C++ class that can perform most of the expected operations with 100% precision. For more precision all you'd need to do is increase the size of the integer or the scaling value. I can upload my c++ code once I get back to my computer. You should probably read up on double though, as it should have enough precision for just about anything, but there are also many factors (order in which operations are performed, number of consecutive operations) that can drastically affect accuracy.

smcd · Oct 11, 2005

You might look into a bignum library - accuracy at the loss of speed.

Doomulation · Oct 12, 2005

zenogais said:
You should probably read up on double though, as it should have enough precision for just about anything, but there are also many factors (order in which operations are performed, number of consecutive operations) that can drastically affect accuracy.

This was what I once thought, of course, but the following code made me change my mind:

Code:

double d1 = 42.0;
double d2 = 10.0;
double d3 = d1 / d2;
// d3 ~= 4.2000000000000000000002

Try it.

smcd · Oct 12, 2005

try googling for "arbitrary precision math library C++" yields results such as:
http://www.tc.umn.edu/~ringx004/article-main.html

Cyberman · Oct 12, 2005

Doomulation said:
This was what I once thought, of course, but the following code made me change my mind:

Code:

double d1 = 42.0; double d2 = 10.0; double d3 = d1 / d2; // d3 ~= 4.2000000000000000000002

Try it.

You've just made a very common error. Most calculator type programs do not use Floating Point. They use BCD numbers with mantissa for computations, and that is the reason why.
The reason you are getting that weird number is you can't exactly represent the number using floating point (which are binary fraction and mantissa) I could get into a long winded discussion why, but I'm not

Cyb

Doomulation · Oct 13, 2005

Cyberman said:
You've just made a very common error. Most calculator type programs do not use Floating Point. They use BCD numbers with mantissa for computations, and that is the reason why.
The reason you are getting that weird number is you can't exactly represent the number using floating point (which are binary fraction and mantissa) I could get into a long winded discussion why, but I'm not

Cyb

Of course I did an error, since the result wasn't right, so that's why I turned here to get an alternative solution. I know nothing of how a real calculator works. Meh. Anyway, that link looks nice.... I'll check out that site.

Search

Search

Precision

Doomulation

?????????????????????????

zenogais

New member

smcd

Active member

Doomulation

?????????????????????????

smcd

Active member

Cyberman

Moderator

Doomulation

?????????????????????????