I think what you should get is, for each fixed $\alpha$, asymptotically $O(N^{1-\alpha})$.
Here's why: do the change of variables $y = x^\alpha$. Your integral becomes
$$ \alpha \int_1^{N^\alpha} y^{\frac{1-\alpha}\alpha} e^{iy} dy $$
Now integrate by parts to lower the exponent on $y$. The first step would be
$$ = \frac{\alpha}{i}\left( N^{1-\alpha} e^{i N^\alpha} - e^i \right) - \frac{1-\alpha}{i} \int_1^{N^\alpha} y^{\frac{1-2\alpha}\alpha} e^{iy} dy$$
Repeating the process $M = \lceil \frac{1}{\alpha} \rceil$ number of times, you get a string of terms that estimates to
$$ = O(N^{1-\alpha} + N^{1-2\alpha} + \ldots + N^{1-(M-1)\alpha}) + O(\int_1^{N^\alpha} y^{\frac{1-M\alpha}{\alpha}} e^{iy} dy) $$
Since $M\alpha \geq 1$, the final integral is dominated by a constant. Noting that for large $N$, the first term inside the first $O(\cdot)$ dominates, you get the result.