I am having trouble understanding Riemann's definition of the zeta function, and I will need to give a brief summary here before I can get to my question.
In his 1859 paper, Riemann derived the integral representation $$\zeta(s)=\sum_{n=1}^\infty\frac{1}{n^s}=\frac{1}{\Gamma(s)}\int_0^\infty \frac{x^{s-1}}{e^x-1}dx$$ that is valid for $\mbox{Re}(s)\gt 1$, and then modified the integral in order to define a function that is defined for all complex values of $s$, except $s=1$, where it has a simple pole. The extension is given by $$\zeta(s)=\frac{\Gamma(1-s)}{2\pi i}\int_C \frac{(-z)^s}{e^z-1}\frac{dz}{z}$$ where $C$ is a "Hankel contour", that is, a path that travels from $+\infty $ at a small distance $\epsilon$ above the positive $x$-axis, circles around the origin once in counterclockwise direction with a small radius $\delta$, and returns to $+\infty$ traveling at distance $\epsilon$ below the positive real axis. Taking the limit as $\epsilon\rightarrow 0$ and $\delta \rightarrow 0$ one can see that the integral $$\int_C \frac{(-z)^s}{e^z-1}\frac{dz}{z}$$ becomes $$(e^{i\pi s}-e^{-i\pi s})\int_0^\infty\frac{x^{s-1}}{e^x-1}dx$$ and then the rest follows easily from known identities satisfied by the Gamma function. While the original real integral over $[0,\infty)$ is clearly divergent if $\mbox{Re}(s)\leq 1$, the contour integral over $C$ is defined for all complex $s$, because the path stays away from the singularity at $s=0$ and from the branch cut along the positive $x$-axis.
My problem is understanding why the integral over $C$ does not depend on $\epsilon$ and $\delta$, so that we can keep them at a safe positive distance from the singularities for the definition, but we can take the limit for the purpose of evaluating the integral. I know that by Cauchy's theorem we can modify a path of integration (without changing the value of the integral) starting and ending at the same point as long as we do not cross any singularity, but this path starts and ends at infinity, so I am not sure how to rigorously proceed using Cauchy's theorem. Even if I start the path at $R+i\epsilon$ and end it at $R-i\epsilon$ for some large $R$, the path starting and ending points change as $\epsilon$ changes.