Directional derivatives & steepest ascent

The slope in a direction v with ‖v‖₂ = 1. For differentiable f it is ⟨∇f(x), v⟩, so by Cauchy–Schwarz the gradient direction gives the steepest ascent, with slope ‖∇f(x)‖.

Chapters

Formulas at a glance · First derivatives

Partial derivatives

\frac{\partial f}{\partial x_{i}} (x) = \lim_{h \to 0} \frac{f (x + h e_{i}) - f (x)}{h}

Gradient & product rule

\nabla f (x) = (\frac{\partial f}{\partial x_{1}} (x), \dots, \frac{\partial f}{\partial x_{d}} (x))

Total derivative

\lim_{y \to 0} \frac{f (x + y) - f (x) - d f_{x} (y)}{‖ y ‖} = 0

Tangent plane & linearization

z = f (x_{0}, y_{0}) + f_{x} (x_{0}, y_{0}) (x - x_{0}) + f_{y} (x_{0}, y_{0}) (y - y_{0})

Directional derivatives & steepest ascent

D_{v} f (x) = ⟨ \nabla f (x), v ⟩, | D_{v} f (x) | \leq ‖ \nabla f (x) ‖

Quadratic forms xᵀCx

\nabla f (x) = x^{⊤} (C + C^{⊤}) = {((C + C^{⊤}) x)}^{⊤}

Directional derivatives & steepest ascent

Working

Partial derivatives

Gradient & product rule

Total derivative

Tangent plane & linearization

Directional derivatives & steepest ascent

Quadratic forms xᵀCx