rounding
- From: "Frank Swarbrick" <Frank.Swarbrick@xxxxxxxxxxxxxx>
- Date: Mon, 1 Oct 2007 17:20:54 -0600
Here is some current production code:
77 ADJ-MEMO-CREDITS PIC S9(7)V99 COMP-3.
77 RSRV-ACC-INT PIC S9(6)V999 COMP-3.
77 RSRV-ACC-INT-ROUNDED PIC S9(6)V99 COMP-3.
COMPUTE ADJ-MEMO-CREDITS ROUNDED =
ADJ-MEMO-CREDITS - (RSRV-ACC-INT + .005).
Example Input:
ADJ-MEMO-CREDITS = 1896.59
RSRV-ACC-INT = 16.082
After calculation:
ADJ-MEMO-CREDITS = 1880.50
However, the expected result is 1880.51
I can do the following to make it work (and possibly easier to follow).
Fix to make it work:
COMPUTE RSRV-ACC-INT-ROUNDED = RSRV-ACC-INT
COMPUTE ADJ-MEMO-CREDITS =
ADJ-MEMO-CREDITS - RSRV-ACC-INT-ROUNDED.
After calculation:
ADJ-MEMO-CREDITS = 1880.51
The point being that the intention of the original code is to round
RSRV-ACC-INT from three to two decimal points and then subtract that result
from ADJ-MEMO-CREDITS. But it does not work the way the original programmer
intended.
Interestingly, I can also do this to make it work:
COMPUTE ADJ-MEMO-CREDITS ROUNDED =
ADJ-MEMO-CREDITS - RSRV-ACC-INT + .005.
The only difference between this and the original is I removed the parens.
What's really interesting is that the original logic looks, to the casual
user, more like it's supposed to work. Because the intention is to subtract
the rounded accrued interest from the memo-credits field.
Obviously this is not working. Using a calculator and doing step by step I
get this:
(Switch to a fixed-size font to make the following readable...)
With parens
1896.59 - (16.082 + 0.005)
1896.59 + 16.087
1880.503
1880.50
Without parens
1896.59 - 16.082 + 0.005
1880.508 + 0.005
1880.513
1880.51
Round first, then subtract
1896.59 - 16.08
1880.51
Anyway, I think I will stick with the 'two step' calculation because I know
(think I know!) that it will always work. I'm not quite convinced that the
version without the parens will always work. I think I've been around
computers too much that I can no longer do simple math!
Now if there was a FUNCTION ROUND that took in a numeric field and a desired
number of decimal point positions (err, whatever), something like this might
work:
COMPUTE ADJ-MEMO-CREDITS =
ADJ-MEMO-CREDITS - FUNCTION ROUND(RSRV-ACC-INT 2).
Frank
.
- Follow-Ups:
- Re: rounding
- From: Frank Swarbrick
- Re: rounding
- From: Louis Krupp
- Re: rounding
- From: Robert
- Re: rounding
- From: Doug Miller
- Re: rounding
- Prev by Date: Re: Taxes
- Next by Date: Re: How proprietary is the "COBOL file system"
- Previous by thread: OT: Taxes
- Next by thread: Re: rounding
- Index(es):