Rounding Error?

[thelebk]

I have been tasked to support a database to track expenses in FM 7.0 that works kind of like an excel spreadsheet. I.e. there is a column of enterable number fields (say e1,e2,e3,e4 etc) and at the bottum is a calc field (Sub) that is defined as e1+e2+e3+e4. The fields are all set to display 2 decimal places, use currency, thousands separator.

What is baffling me is that FM cannot seem to add properly on some records. Example e1=0, e2=0, e3=$63.75, e4=$63.75, Sub=$128. Where the heck is the extra $.50 coming from? 63.75+63.75= 127.5 not 128. If it is rounding the .75 up to .8 the total should be 127.6, but it not. Perhaps it is rounding the calculated result of 127.5 to 128. If so how do I turn rounding off? In financial calculations even a single penny is significant... I erased the data and re-entered it, but get the same result.

Does anyone know what is happening and how to fix it?

[Lee Smith]

From Rounding?

.75 rounds up to the next whole number.

In stead of Round (YourField; 0)

Try Round (YourField; 2)

[thelebk]

it did not round the .75 to .8 or the calc result would have been 127.6. The calc result is actually 128. Thus it appears to not like to display 127.5 even though two decimal places are specified, and my guess is that it is rounding to 128 to avoid it.

Please note I am not using the round function. I do not want rounding. I want precise numbers, even one penny off will hose my calculations.

The calc is sub=e1+e2+e3+e4. For some reason FM cannot perform this simple addition task accurately. I am guessing, and it is only a guess, that this may be a rounding problem. Another example:

when I try to add $41.39 + $41.39, FM returns the calculated total of $82.77. This is clearly wrong as these figures add to $82.78. Any thoughts on how to fix this?

[Lee Smith]

It would be easier to see what you are doing Right or Wrong if you would attach a copy of your file, or make a demo of this.

Lee

[comment]

A sample file demonstrating these errors would be useful.

Also, are you patched up to v.7.0v3?

[Vaughan]

Check what's actually been entered into the fields.

Number fields can be set to display numbers with fixed decimal places and FMP displays these numbers rounded, but continues to use the full precision for calculations.

[thelebk]

I figured out what was happening. FM Was doing the subtotal calculation in floating point keeping half pennies etc. When the figures in the column displayed in two decimal places however FM rounded them causing the descrepancy.

Thanks.

[David Jondreau]

That makes sense, except your example was 4 items that resulted in a .50 difference. That's not a half penny x 4 difference.

[Lee Smith]

That is why I suggested a copy, or a demo of what the member is trying to do.

Saves a lot of guess work.

Lee