This seems to work:
dzeta[x_] := Derivative[1, 0][Zeta][y, x]
Plot[{Exp[dzeta[0, x] + 1/2 Log[2 Pi]] , Gamma[x]}, {x, -3, 3}]
The derivative of a function returns a function. I think that in your version, maybe Mathematica binds x to specific numerical values during the plot and then tries to take the derivative. The version above computes the derivative once and for all and then does the plot.
EDIT:
Oops, Annix is right. This is not the first time I have been undone by mistaking the graph color for two graphs instead of one. Anyway, I'm sorry for the false starts, but the following code properly displays both functions and the graphs coincide.
Clear[x, y, dzeta]
dzeta[y_, x_] := Derivative[1, 0][HurwitzZeta][y, x]
Plot[{Exp[dzeta[0, x] + 1/2 Log[2 Pi]], Gamma[x]}, {x, -3, 3}]
Changing "Zeta" to "HurwitzZeta" in your original expression also works:
Plot[{Exp[
Derivative[1, 0][HurwitzZeta][0, x] + 1/2 Log[2 Pi]]}, {x, -3, 3}]