Re: How to calculate Hessian Matrix

On 5 août, 09:39, fj <francois.j...@xxxxxxx> wrote:
On 5 août, 09:32, fj <francois.j...@xxxxxxx> wrote:

On 5 août, 04:45, John <john131...@xxxxxxxxx> wrote:

Hi,

Does anyone know where there is any function/subroutine can compute
the Hessian Matrix of a multivariable function? Thanks a lot!

John

First of all, your are not posting in the right group.

It is very easy to compute an Hessian matrix using numerical
derivatives. The only problem is the cost especially if the multi
variable function is itself a "vector" function.

The simplest way consists in writing a routine computing the gradient.
After that, you write the function computing the Hessian in the same
way. Let us assume the the function is only scalar :

double precision function f(n,x)

subroutine hessian(n,x,f,h)
implicit none
integer,intent(in) :: n
double precision,intent(in) :: x(n)
double precision,intent(out) :: h(n,n)
double precision :: xx(n),df0(n),df(n),xx0
double precision,parameter :: epsilon=1.d-5
integer :: j
xx=x
do j=1,n
xx0=xx(j)+epsilon
h(:,j)=(df-df0)/epsilon
enddo
end subroutine

Of course, you might improve this simple solution : the parameter
"epsilon" may be passed as argument and may be a vector. You might
also prefer central derivatives rather than "right" derivatives and
define routines f and gradient using interfaces rather than EXTERNAL
declaration ...

Oups ! The instructions in the loop are wrong and should be replaced
by :

do j=1,n
xx0=xx(j)
xx(j)=xx0+epsilon
h(:,j)=(df-df0)/epsilon
xx(j)=xx0
enddo

Sorry, I was not awake !

Just another remark : as the Hessian matrix is symmetrical (d/dxi(df/
dxj)=d/dxj(df/dxi)) the previous algorithm is not optimal. But I often
use it for computing numerically the Jacobian matrix of a vector
function (the gradient is a kind of vector function) in order to
verify that a more efficient (but also much more complicated) version
using formal derivatives has no mistake.

Notice also that it is rare to need the Hessian and even rarer to
compute it explicitly. An Hessian approximation is often sufficient
and algorithms exist which are able to estimate it. For instance on
Wikipedia :

http://en.wikipedia.org/wiki/BFGS
.

Relevant Pages

• Re: How to calculate Hessian Matrix
... you write the function computing the Hessian in the same ... also prefer central derivatives rather than "right" derivatives and ... But the problem was I found it's quite tricky on choosing the epsilon. ... Now this is a little bit more complicated in your case if the gradient ...
(comp.lang.fortran)
• Re: fmincon
... That would make a lot of sense, but according to the Matlab help DerivativeCheck only looks at the gradient. ... Hessian he finds non-zero elements almost everywhere. ... derivatives do not match up in its DerivativeCheck. ... the first derivatives and it did not give me any ...
(comp.soft-sys.matlab)
• Re: fmincon -> Hessian -> Covariance matrix
... Now, although I am trying to find out what the gradient and hessian over b0 b and sd variables is, I still feel like I have stuck. ... You just need to take derivatives and use the chain rule a few times. ...
(comp.soft-sys.matlab)
• Re: Hessian of vector-valued functions
... is calculated or point me towards a reference where the ... I understand the array of second partial ... derivatives is not a matrix, but a tensor of rank 3.... ... In that context, `hessian'  means nothing ...
(sci.math)
• Re: How to calculate Hessian Matrix
... It is very easy to compute an Hessian matrix using numerical ... double precision function f ... subroutine gradient ... also prefer central derivatives rather than "right" derivatives and ...
(comp.lang.fortran)