Qiaochu's answer is simpler, but the following almost classifies all groups of order $2p$, so I'll leave it here as well.
Lemma 1: The order of an element divides order of the group.
No new proof, but easy enough.
Lemma 2: There exists an element of order $p$.
Can circumvent usual proof as follows: If there is an element of order $2p$ G is cyclic. If no element of order $p$ or $2p$ then $a^2=e$ for all $a$, so $G$ is abelian ($abab=e$, so $ab=b^{-1}a^{-1}=ba$). Then either use the fact that you know all abelian groups, or the fact that then $G$ is a vector space over the field $Z_2$, so has basis and so has order $2^k$.
Call this element $a$.
Lemma 3: Any subgroup of index 2 is normal.
There are two cosets, and one of them is the subgroup itself. So right and left cosets coincide.
Applying this to the subgroup generated by $a$, for $b$ not a power of $a$, we have $bab^{-1}=a^m$. The group contains only $a^k$ and $ba^l$'s, which are all distinct. If $m=1$ they all commute and $ba$ generates the cyclic group. If $m\neq1$ there is no center. (The only step in classification left is to show that different $m \neq 1$ lead to the same group, which is not hard either.)