# Rounding Error?

## Recommended Posts

I have been tasked to support a database to track expenses in FM 7.0 that works kind of like an excel spreadsheet. I.e. there is a column of enterable number fields (say e1,e2,e3,e4 etc) and at the bottum is a calc field (Sub) that is defined as e1+e2+e3+e4. The fields are all set to display 2 decimal places, use currency, thousands separator.

What is baffling me is that FM cannot seem to add properly on some records. Example e1=0, e2=0, e3=\$63.75, e4=\$63.75, Sub=\$128. Where the heck is the extra \$.50 coming from? 63.75+63.75= 127.5 not 128. If it is rounding the .75 up to .8 the total should be 127.6, but it not. Perhaps it is rounding the calculated result of 127.5 to 128. If so how do I turn rounding off? In financial calculations even a single penny is significant... I erased the data and re-entered it, but get the same result.

Does anyone know what is happening and how to fix it?

##### Share on other sites

From Rounding?

.75 rounds up to the next whole number.

In stead of Round (YourField; 0)

Try Round (YourField; 2)

##### Share on other sites

it did not round the .75 to .8 or the calc result would have been 127.6. The calc result is actually 128. Thus it appears to not like to display 127.5 even though two decimal places are specified, and my guess is that it is rounding to 128 to avoid it.

Please note I am not using the round function. I do not want rounding. I want precise numbers, even one penny off will hose my calculations.

The calc is sub=e1+e2+e3+e4. For some reason FM cannot perform this simple addition task accurately. I am guessing, and it is only a guess, that this may be a rounding problem. Another example:

when I try to add \$41.39 + \$41.39, FM returns the calculated total of \$82.77. This is clearly wrong as these figures add to \$82.78. Any thoughts on how to fix this?

##### Share on other sites

It would be easier to see what you are doing Right or Wrong if you would attach a copy of your file, or make a demo of this.

Lee

##### Share on other sites

A sample file demonstrating these errors would be useful.

Also, are you patched up to v.7.0v3?

##### Share on other sites

Check what's actually been entered into the fields.

Number fields can be set to display numbers with fixed decimal places and FMP displays these numbers rounded, but continues to use the full precision for calculations.

##### Share on other sites

I figured out what was happening. FM Was doing the subtotal calculation in floating point keeping half pennies etc. When the figures in the column displayed in two decimal places however FM rounded them causing the descrepancy.

Thanks.

##### Share on other sites

That makes sense, except your example was 4 items that resulted in a .50 difference. That's not a half penny x 4 difference.

##### Share on other sites

That is why I suggested a copy, or a demo of what the member is trying to do.

Saves a lot of guess work.

Lee

## Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

×   Pasted as rich text.   Paste as plain text instead

Only 75 emoji are allowed.

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×

×
×
• Create New...