# Introduction

Optimization is a focus on many kinds of machine learning algorithms like Linear Regression, SVM and K-means. But actually many kinds of target function is non-convex which means we can only find its local minima. But convex functions still plays an important role in machine learning. And Hessian Matrix is a great algebra tool to analyze convex functions since in most cases our target function will be real, continuous and $2^{nd}$-order differentiable. The main goal of this article is to record the proof of the equivalence between Convex Functions and their Hessians. Here is the some important definitions.

## Convex Set

A **Convex Set** $C\subseteq \Re^n$ is a set of points s.t. $\forall x, y \in C$ and $t \in [0,1]$, $tx+(1-t)y \in C$.

## Convex Function

A function $f:\Re^n \rightarrow \Re$ is a **Convex Function** if for $x, y \in D$, where $D$ is a **convex set**, $f$ and any $t \in [0,1]$ makes

## Hessian Matrix

A **Hessian Matrix** is a square matrix of **second-order partial derivatives** of a function $f:\Re^n \rightarrow \Re$, usually written as:

## Positive Definite/Semi-Definite Matrix

A **real symmetric matrix** $P$ is called **Positive Semi-Definite** (PSD) when for all $x \in \Re^n$, there are $x^TPx \geq 0$. And it’s called **Positive Definite** (PD) when for all $x \neq 0 \in \Re^n$, there are $x^TPx > 0$.