For linear programming, it's well known that the dual of the dual is the primal. I'm wondering if it is the case for Lagrange duality, and I'm having a hard time showing this.
Notationally, let the primal problem be:
$$\text{minimize } \quad f_0(x)$$ $$\text{subject to } \quad f_i(x) \leq 0, \quad i = 1, \dots, m$$
And the dual be:
$$\text{minimize } \quad -g(\lambda) = - \inf_x L(x, \lambda)$$ $$\text{subject to } \quad -\lambda \leq 0$$
Where $L(x,\lambda) = f_0(x) + \sum_{i=1}^m \lambda_i f_i(x)$ is the Lagrangian.
I suspect it isn't true in general that the dual of dual is the primal. However, intuitively when I hear the term dual I assume that the dual of the dual should be the primal, so this got me confused.