Re: Sybase convert oddity


rickpoleshuck@xxxxxxxxxxx wrote:
select convert(numeric(16,2), 5.325),convert(numeric(16,2), '5.325')

returns

5.32 5.33


I would like to understand the logic behind this behavior, if any.

Hi. This has nothing to do with Java, so you
should post it to comp.databases.sybase.
It probably has to do with the fact that a
conversion to numeric will use an exact
algorithm for it's input. The string input is
exact, but when you pass 5.325 directly,
the DBMS's SQL parser will convert that to
an IEEE float which may not be able to exactly
represent a decimal 5.325. The exact value
may be slightly less, leading to the round-down.

Joe Weinstein at BEA Systems

.

Relevant Pages

  • Re: Sybase convert oddity
    ... The string input is ... exact, but when you pass 5.325 directly, ... the DBMS's SQL parser will convert that to ... Lee Fesperman, FFE Software, Inc. ...
    (comp.lang.java.databases)
  • Re: Sybase convert oddity
    ... The string input is ... exact, but when you pass 5.325 directly, ... the DBMS's SQL parser will convert that to ... non-standard behavior by the DBMS. ...
    (comp.lang.java.databases)
  • Re: Sybase convert oddity
    ... exact, but when you pass 5.325 directly, ... the DBMS's SQL parser will convert that to ... an IEEE float which may not be able to exactly ...
    (comp.lang.java.databases)