Suppose I have a variable $s$ that has a geometric distribution with success parameter x. So the probability of success on trial $s$ is
$p_s = (1 - x)^{s - 1} x$,
Consider the following function of $s$
$V(s) = \delta^s$,
where $\delta < 1 $. Then the expected value of this function is given by
$\sum_ 1^{\infty} V(s) p_s = \frac{\delta x} {1 - \delta (1 - x)}$.
Consider now the deviation of V(s) from its mean :
$(V(s) - \frac{\delta x} {1 - \delta (1 - x)}) $
What distribution does this have?
What I am really interested in is the behavior of the follow ordinary stochastic differential equation:
$\dot{V} = \delta^s - V$,
stochastic because the law of motion depends on the random variable $s$. The stochastic approximation technique allows me to focus on the mean dynamics, which are given by
$\dot{V} = \frac{\delta x} {1 - \delta (1 - x)} - V$,
whose equilibrium is simply $\hat{V} = \frac{\delta x} {1 - \delta (1 - x)}$.
I would like to characterize the large deviation properties of the original ODE, IE calculate the exponential likelihood $V$ crosses a particular boundary $c$.
For example, if $\delta = .95$ and $x= .01$, $\hat{V} = 0.159664$. If we set $V_0 = 0.159664$, and let $V_t = V_{t-1} + .2 (\delta^s - V_{t-1})$, and $s$ has the above distribution, how do I calculate the expected time to $V_t$ crossing, say, $c=.4$? What is the associated rate function?
Editing to give the background to the problem:
I am interested in the following stochastic dynamic system:
$V_t = V_{t-1} + \gamma (\delta^{S_{t-1}} - V_{t-1})$,
where $\delta <1$, $\gamma<1$, both positive, and each $S_t$ is an i.i.d. geometric random variable with success parameter $x$. What this models is agents who have to wait a geometric length of time to get a reward of value 1, and they have time discount factor $\delta$, and they are learning the expected value of their reward using a constant gain adaptive learning procedure, with gain $\gamma$.
So considering $S$ as following a Poisson, rather than Geometric, makes little difference to the sense of the problem. The goal is
Find the equilibria of the learning dynamics
Characterize the mean time to escape from this equilibrium, the large deviation properties.
So, for large deviation properties, I should be able to use something like Cramer's theorem, I think. As Mike says below, iterating the dynamics gives
$V_t = (1−γ)^tV_0+γ\sum_{k=0}^{t−1}(1−γ)^{t−1−k}δ^{S_k}$
So it all depends on $\sum_{k=0}^{t−1}(1−γ)^{t−1−k}δ^{S_k}$, the discounted sum of some random variables.
The Probability that $V_t >c$ is then given by
$Pr(V_t > c ) = Pr(\sum_{k=0}^{t−1}(1−γ)^{t−1−k}δ^{S_k} > \frac{c-(1−γ)^tV_0}{\gamma})$
Let $Z:=\frac{c-(1−γ)^tV_0}{\gamma}$. so we need the distribution of $\sum_{k=0}^{t−1}(1−γ)^{t−1−k}δ^{S_k}$, which as Mike says is a discounted sum of independent random variables, and so we have that this is approximately normal, for large $t$.
Turning now to the question of large deviations, if things were not discounted by powers of $(1-\gamma)$, we could appeal directly to Cramer's Theorem;
$ Pr(\sum \delta^{S_k} > n a) \leq Exp[-n(r^* a - Log(E[Exp(r^* \delta^S)])] $
where $r^* $ is chosen to maximize $r a - Log(E[Exp(r \delta^S)]$. The problem would be to simply calculate the moment generating function of $\delta^S$. But, I don't exactly want $Pr(\sum \delta^{S_k} > n a)$, I need $ Pr(\sum (1-\gamma)^{t-1-k} \delta^{S_k} > a)$; so not an average of $\delta^{S_k}$s, but a discounted sum. so it is not quite a direct application.