Re: Regression analysis -- how-to?

From: Joseph McDonnell (mcdonnell_at_bmg.eur.nl)
Date: 01/11/05 

Date: Tue, 11 Jan 2005 10:18:42 +0100

korax1214@mailandnews.co.uk wrote:

> (Yes, I know that sci.math would be more appropriate for this thread --
> were it not for the fact that, the last time I asked an algorithm
> question there, the newsgroup's troll replied with a post which started
> "there's no such thing as rectangular or polar complex numbers, there
> are only complex numbers" (which may be true of *pure* maths[1]
> (although I won't believe this until I get confirmation from someone
> intelligent) but is definitely *not* true of *applied* maths (e.g.
> algorithms) where the distinction between rectangular and polar forms
> is very real (pun not intended) and quite important. For instance, the
> algorithm for multiplying two rectangular complex numbers is very
> different from the algorithm for multiplying two polar complex numbers,
> just as the algorithm for multiplying 1/2 by 3/4 is very different from
> the algorithm for multiplying 0.5 by 0.75, even though the same pair of
> fractions are involved in both cases), and went rapidly downhill from
> there. (The idiot in question obviously not only doesn't know what
> "twit" means, at least not in British English which is what I speak and
> write (it's *not* an insult), but from his reply to my reply to him, he
> obviously doesn't even know what "fraction" means.) Hence I am posting
> this to a group where I am likely to get *some* intelligent replies --
> besides, I already tried sci.math with this question, and it appears
> that nobody there is capable of answering it.)
>

You've forgotten Monty Python's Upper Class Twit of the Year Contest.

>
> Could anyone give me one or more algorithms for performing regression
> analysis, calculating the coefficient of correlation of a dataset, or
> any related topic of interest? Or give me a pointer as to where
> (preferably online) I could find this information?
>

May I ask why you want to do this? Do you want to learn about regression?
Then I suggest you consult a few books. There are lots of books which will
provide you with theory, which, for linear regression, is fairly simple but
is much less so for other forms of regression. Trying to deduce what's
going on from code would be difficult and difficult to generalise to a new
situation.

>
> I know that there are at least six types of regression analysis
> (linear, quadratic, logarithmic, exponential, power and inverse) and at
> least two types of linear correlation coefficient (rank and product
> moment, IIRC), but I'd like to know the details...
>

I suspect what you mean by 'quadratic' is the inclusion of an x^2 term as
well as an x term. If so, you're still talking about linear regression
('linear' here means linear in the regression parameters, not in the x's).
This might also be the case for the other forms you mention or you could be
talking about different 'links' e.g. y = log(a + b*x). The theory for these
kinds of models is a little more difficult, but you seem to have a maths
(note the spelling) background. I'd recommend Annette Dobson's
"Introduction to Generalized Linear Models".

>
> I know how to calculate the arithmetic mean and the product/population
> standard deviations of a dataset, which I think are at least 70% of
> what I need to do linear regression analysis, but I have no idea how to
> go beyond this.
>

Basically you also need to calculate the correlation coefficients between
the variables and you have all you need.

>
> [1] Please don't tell me that the "correct" spelling is "math" (British
> English, remember?); "mathematics" is a plural-only word, so one can't
> have only one math, any more than one can have only one applau. :-)

Even worse, them Yanks say "TOM-MAY-TOE." :-)

Regards

Joseph

Relevant Pages

  • Re: Correlation coefficient not suited for small samples!? Cross validation as an alternative?
    ... correlation coefficient tends to increase strongly with decreasing ... sample size and that therefore it is not an adequate measure of linear ... Regression coefficients have less dependence on variance, ...
    (sci.stat.consult)
  • Re: Calculating two free axes
    ... You can permuate the coordinates if vn is zero to get ... On paper you solve systems of linear equations easily. ... generate a set of linearly independent vectors huh? ... algorithm to do this while I have. ...
    (comp.graphics.algorithms)
  • Re: interaction terms in regression model
    ... approach to regression than that in some other disciplines. ... centering constants you use (I'm assuming a two-variable model here... ... two linear and one interaction between those) the "significance" of the ... The "failure to center" ...
    (sci.stat.edu)
  • Re: System Identification
    ... the sum-of-sines contains N/2 sinusoidals. ... the first step of his algorithm. ... which essentially computes the linear prediction coefficients from the ... needs to use the modified covariance equations, ...
    (comp.dsp)
  • Re: Paper & pencil password algorithm
    ... You should find the definition of linear on the internet and look at ... of the digits with an additional one added in on the leftmost digit. ... addition and non-linear substitution ... You asked me about the timing of the individual parts of the algorithm. ...
    (sci.crypt)