Re: Rounding error when converting from double to int
- From: gordonb.9dw2o@xxxxxxxxxxx (Gordon Burditt)
- Date: Sat, 15 Aug 2009 14:19:02 -0500
Do your calculation that generates the fractions in integers. I'm
assuming here that negative numbers are not involved. For example,
if you have $9.51 (represented as 951 cents) and you need to take
8.875% of it (suppose it's a sales tax calculation), multiply 8.875%
by one million and get 88750. (This presumes that sales tax rates
are always multiples of 0.0001%). Be sure *NOT* to put 8.875% into
a double. Represent it as 88750 / 1000000, and treat numerator and
first of doing this division, from 8.875 -> 88750 is there one rounding?
I told you above *NOT* to put the percentage into a double in the first
place, so there is no rounding.
if for example we have 8.795% and it is rappresented in double like
8.79499999999999999
DO NOT put that value into a double.
the result afther the "->" will be 87949 of 87950?
The multiplication should be done in mathematics, not C.
.
- Follow-Ups:
- References:
- Rounding error when converting from double to int
- From: mark
- Re: Rounding error when converting from double to int
- From: Gordon Burditt
- Re: Rounding error when converting from double to int
- From: io_x
- Rounding error when converting from double to int
- Prev by Date: HOW TO PRODUCE LIGHT {Noor(Divine Energy)} IN YOUR HEART.
- Next by Date: Re: Richard Heathfield
- Previous by thread: Re: Rounding error when converting from double to int
- Next by thread: Re: Rounding error when converting from double to int
- Index(es):
Relevant Pages
|
Loading
