RE: prePEP: Decimal data type

From: Tim Peters (tim.one_at_comcast.net)
Date: 11/05/03 

Date: Tue, 4 Nov 2003 19:57:31 -0500
To: "Python-List \(E-mail\)" <python-list@python.org>

[Batista, Facundo]
> Nice picture, :). But in Money you *want* to get rounded in the
> decimal places.
>
> Example of decimal with precision = 100:
>
> 1122334455667788 --> 1122334455667788
> 1122334455.667788 --> 1122334455.667788
>
> But in Money, you may want to get rounded to two decimals:
>
> 1122334455.667788 --> 1122334455.67

What are these example of? Literals? If you don't want 40 digits after the
decimal point in a literal, don't write 40 digits after the decimal point in
a literal. Or, if you have to, use the spec's quantize() function to round
to the number of fractional decimal digits you want. After that, + and -
take care of themselves (they neither increase nor decrease the number of
decimal digits after the radix point, and IBM's arithmetic *remembers*
that -- e.g., 3.12 - 1.12 is 2.00 in that scheme, not 2 or 2.0 or 2.000).
You do need to explicitly round (if desired) after * and /. Because of the
high value attached to 100% conformance to imposed standards in accounting
apps, I think it's a good thing that you're forced to round explicitly when
the result of a calculation can't be represented exactly in the mandated
format -- those are exactly the places where your app will fail to meet the
requirements, so it's good to be aware of them.

> ...
> So, unbounded precision and fixed point are the same? (I always
> assumed that)

It depends on who you ask, but most people would probably say "no". "Fixed
point" has no fixed meaning -- it's been applied to a plethora of schemes
over the decades, and each meaning has its own gang of frightened defenders
<0.9 wink>. My FixedPoint class provided an historically peculiar meaning
(fixed # of digits after the decimal point, unbounded # of digits before
it), but one I found useful at the time. Most meanings of "fixed point"
have in mind a fixed number of digits both before and after the decimal
point, and rarely the same number of digits before as after. Variants in
all areas are rampant.

For example, for multiplication Java's BigDecimal class gives a number of
digits after the decimal point in the product equal to the sum of the number
of digits after the decimal points in both inputs; in the same case,
FixedPoint rounds the infinitely-precise result to the larger of the number
of digits after the decimal point across both inputs.

Both schemes are crazy for some apps. For example, if there's 8.875% tax on
a $1.01 item,

    1.08875 * 1.01 = 1.0996375 # BigDecimal
                   = 1.09964 # FixedPoint

are both probably useless as-is. Under the IBM spec, you multiply and then
call quantize with a second argument of (for example) 0.01. Assuming
banker's rounding, that gives the 1.10 you're probably seeking. Or maybe
that's not what you want -- maybe your municipality wants the excess
truncated. They're both equally easy (and equally difficult <wink>) under
the IBM spec. What doesn't work is hoping that some class will do rounding
correctly in all cases by magic.

Relevant Pages

  • Re: check decimal and if numeric
    ... so effectively truncates an subsequent digits for positive numbers. ... It's different when you round negative numbers down, ... decimals, who's to say the third or fourth digit was the one that was in error? ...
    (microsoft.public.access.formscoding)
  • Re: Truncating Numbers
    ... digits to the right of the decimal. ... decimals), but the number is still 12.3546. ... extra 2 digits at the end can cause the "across" sums ... There are 2 methods to round the computation. ...
    (microsoft.public.access.queries)
  • Re: Time Accuracy
    ... That means the 864,000 tenths of a second in one day can theoretically be handled by the available decimals. ... Perhaps I should have said unreliable digits rather than unreliable numbers. ... oddities of IEEE floating point representation. ... raised issues about the resolution of the Excel NOWfunction. ...
    (microsoft.public.excel.misc)
  • Re: Math issue
    ... <it is common to almost all computer software and hardware> ... There are implementations of the COBOL and other languages that do not suffer from this problem, because they use either scaled integer representations internally or true binary representations of numbers. ... We should be careful with descriptions like "digits beyond the 15th". ... If I re-format to number and expand the decimals ...
    (microsoft.public.excel)
  • Re: Rounding down with a calculation
    ... Have you tried formatting the cellto a number with 2 decimals? ... "Sally Sibthorpe" wrote in message ... How do I round the result down to 2 digits? ...
    (microsoft.public.excel.worksheet.functions)
Loading