The symbol $\circ$ is used for composition of functions: $g\circ h(x)$ means $g(h(x))$. For composing a function with itself, many people use $f^{\circ n}(x)$, defined inductively by
\begin{align*}
f^{\circ 1}(x) &= f(x),\\
f^{\circ(k+1)}(x) &= f\Bigl( f^{\circ k}(x)\Bigr).
\end{align*}
So $f^{\circ 2}(x) = f(f(x))$; $f^{\circ 3}(x) = f(f^{\circ 2}(x)) = f(f(f(x)))$, etc.
Timothy Wagner also notes that $f^n(x)$ is often used; that is true, but it has the disadvantage of often being ambiguous. For example, $\sin^2(x)$ is (almost) always understood to mean $(\sin(x))^2$, and not $\sin(\sin(x))$.