I am not sure whether you meant initially at rest relative to the universe, or to the surface of the Earth. Here are the answers to both versions:
Universe
Let the latitude be $\theta_0$. In the non-rotating reference frame (of the universe), the motion of the stone is in simple harmonic motion. So $r(t) = r_0 \sin (\omega t)$, where $2\pi / \omega$ is the time it takes to get to the center of the earth. The latitude is constant. And the longitude $\phi_0$ (in the non-rotating reference frame) is also constant. But the earth spins with constant angular velocity $\Omega_0 = 2\pi$ radians per 24 hours. So in the rotating coordinate $\phi'(t) = \phi_0 - \Omega t$.
So measured relative to the surface of the earth in spherical coordinates you have the parametric description
$$ r(t) = r_0\sin(\omega t) $$
where $r_0$ is the radius of the Earth. (Note: this assumes that the Earth is perfectly spherical and of uniform density inside, which is obviously not quite physical. Of course, digging a tunnel like that is also not quite physical...)
$$ \theta(t) = \theta_0, \phi(t) = \phi_0 - \Omega t $$
Finding $\omega$ from the assumption that Earth is perfectly spherical and of uniform density is left as an exercise to the reader.
A second interesting exercise is to find the conditions on the density of Earth and on the rate of revolution that allows for the stone to travel in a closed orbit (again, this is much much simpler when considered in the non-rotating reference frame...)
Surface of the earth
Again we work in the non-rotating reference frame. We have again, via the constant density assumption, (and assuming that the mass of stone is 1) that the potential energy is $P = \alpha r^2$. The kinetic energy is $2K = \dot{r}^2 + r^2\dot{\theta}^2 + (r\cos\theta)^2\dot{\phi}^2$. The conservation of angular momentum means that angular component of the velocity is of size $L / r$, where $L$ can be computed from the rate of rotation of the Earth. So the conserved energy is
$$ E = \dot{r}^2 + \frac{L^2}{r^2} + \alpha r^2 $$
This gives an ODE for $r$. Similarly using a tilted coordinate system you can solve for the angles using the conservation of angular momentum by integrating an ODE and plugging in the solution for $r$.
For the maximum depth, however, you don't need to explicitly solve the ODE: The energy is known initially: $E_0 = 0+ \frac{L^2}{r_0^2} + \alpha r_0^2$, where $L$ depends only on the rate of revolution for Earth, and $\alpha$ on the mass of Earth (assuming uniform density). $r_0$ is the radius of Earth. At the maximum depth, $\dot{r}$ is again 0. So you are down to finding the "other" positive root of the quartic polynomial
$$ E_0 r^2 = L^2 + \alpha r^4 $$
which you can solve explicitly using the quadratic formula
$$ r^2 = \frac{E_0 \pm \sqrt{ E_0^2 - 4 \alpha L^2}}{2\alpha } $$
where the + solution is the radius of the earth, and the - solution is the depth.
Plugging in physical numbers: at the initial drop, $\dot{\theta} = 0$ and $\dot{\phi} = \Omega = \frac{2\pi}{86400} s^{-1}$. The radius of Earth we take to be $6.38 \times 10^6 m$, the mass $6\times 10^{24} kg$. So $\alpha = G M / r_0^3 = 1.5 \times 10^{-6} s^{-2}$.
The conserved angular momentum is initially
$$ L_0^2 = r_0^4 \cos(\theta_0)^4 \Omega^2 = 8.8\times 10^{18}\cos\theta_0^4 m^4 s^{-2} $$
and the conserved Energy is initially
$$ E_0 = L_0^2 r_0^{-2} + \alpha r_0^2 = (\cos\theta_0)^4 2.15 \times 10^5 + 6.1\times 10^7 m^2 s^{-2} \sim 6.1\times 10^7 m^2 s^{-2} $$
(the initial angular momentum contribution is very small).
Now $E_0^2 = 3.7 \times 10^{15}$ and $4\alpha L_0^2 = 5.28\times 10^{13}\cos(\theta_0)^4$, so the answer is that the maximum depth is very close to the center of the earth! Using the binomial expansion we get that
$$ r^2 \sim \frac{L_0^2}{E_0} \implies r \sim 3.9\times 10^5 \times \cos(\theta_0)^2 m$$
So if you start at the equator where $\theta_0 = 0$, the hole will get about 94% to the center of the Earth.
