Newton Raphson and quasi-Newton methods

The simplest second derivative method is Newton-Raphson (NR). In a system involving N degrees of freedom a quadratic Taylor expansion of the potential energy about the point ${\bf q}_k$ is made, where the subscript $k$ stands for the step number along the optimization.

$$E({\bf q}_k+\Delta {\bf q}_k)=E({\bf q}_k)+{\bf g}^T_k \Delta {\bf q}_k+\frac{1}{2}\Delta {\bf q}_k^T{\bf H}_k \Delta {\bf q}_k$$ (2.74)

The vector $\Delta {\bf q}_k=({\bf q}_{k+1}-{\bf q}_k)$ describes the displacement from the reference geometry ${\bf q}_k$ to the desired new geometry ${\bf q}_{k+1}$, ${\bf g}_k$ is the first derivative vector (gradient) at the point ${\bf q}_k$ and ${\bf H}_k$ is the second derivative matrix (Hessian) at the same geometry. Under the approximation of a purely quadratic PES, and imposing the condition of a stationary point ${\bf g}_k=0$ we have the Newton-Raphson equation that predicts the displacement that has to be performed to reach the stationary point.

$$\Delta {\bf q}_k=-{\bf H}^{-1}_k {\bf g}_k$$ (2.75)

Because the real PES are not quadratic, in practice an iterative process has to be done to reach the stationary point, and several steps will be required. In this case the Hessian should be calculated at every step which is high computationally demanding. A variation on the Newton-Raphson method is the family of quasi-Newton-Raphson methods (qNR), where an approximated Hessian matrix ${\bf B}_k$ (or its inverse) is gradually updated using the gradient and displacement vectors of the previous steps. In section 1.3.4.4 we will summarize these methods to update the Hessian.