Re: Rounding of integer divisions (was Re: linear interpolation / Assembler / ATMega32)

From: dwight elvey (dkelvey_at_hotmail.com)
Date: 06/15/04 

Date: 14 Jun 2004 17:48:02 -0700

Ville Voipio <vvoipio@kosh.hut.fi> wrote in message news:<i3k3c4zg27a.fsf_-_@kosh.hut.fi>...
> Paul Keinanen <keinanen@sci.fi> writes:
>
---snip---
>
> The worst thing you may do is to unsignedify the number, round
> it down, and then put the sign back. Because:
>
> -17 / 16 -> -(17 >> 4) -> -1
> -16 / 16 -> -(16 >> 4) -> -1
> -15 / 16 -> -(15 >> 4) -> 0
> ...
> 0 / 16 -> ( 0 >> 4) -> 0
> ...
> 15 / 16 -> (15 >> 4) -> 0
> 16 / 16 -> (16 >> 4) -> 1
>
> The function becomes dead around zero! The result is zero over
> two units, and everything linear becomes non-linear. Cross-over
> distortion sets in...
>
> Unfortunately, this is exactly what C does when you write:
>
---snip---

Hi
 Most high level languages round towards zero, mostly
because it is traditional. Doing floored rounding makes
more sense but even that has to be done carefully.
In something like DSP, one can accumulate a DC offset
because of floored rounding. As you point out, you
get a cross over distortion for rounding towards zero
so there is no absolutely right way to do this.
One has to look carefully at the application one intends
to use it in before determining which way to go on this.
One might even use both methods in the same stretch of code.
 I once designed a XY table that requires 24 bit math
that a fellow was doing 16 bit math by simply extending
it to 32 bits while using the round towards zero.
 It was a disaster. The table would occasionally jump
by a sizable amount while incrememnting by a joy stick.
This was a table that should have positioned to about
two 10/1000th of an inch. This was traced to his round
towards zero math in the Pascal he was using to do the
extended math.
 It would have been nice if the uP makes had included
both types in the hardware so we could choose which
to use and where.
Dwight

Relevant Pages

  • Re: Only 6 million listeners?
    ... but the showbiz rules of math allow you to round _up_ ... for any number that doesn't end with a zero. ...
    (alt.fan.howard-stern)
  • Re: xelatex (and lualatex) error with setmathfont{Cambria Math}
    ... Undefined control sequence. ... l.7 \setmathfont{Cambria Math} ... Missing number, treated as zero. ...
    (comp.text.tex)
  • Re: prePEP: Decimal data type
    ... I think it's a good thing that you're forced to round ... >is about rounding rules and it's quite easy to avoid the "fail to meet the ... ISTM these things are 'way beyond the concern of a math package per se, ... for the meaning of "context" in this PEP context, ...
    (comp.lang.python)
  • Re: Is time continious or descrete?
    ... With math, science related math, numbers are expected. ... There are stated philosophical justifications for all of Physics. ... smallest possible scales, the zero of physics. ...
    (sci.physics)
  • Re: Absolute vs Incremental 3D programming
    ... Fanuc 18i control for which we do not have a post processor which ... never posted 3D tool paths in anything but absolute, ... deviation from zero will still be in the .015in window. ... I'll also mention that the CNC control does math in AT LEAST six ...
    (alt.machines.cnc)