So let's see from where does the centripetal force comes from.
Imagine we have a body that moves in 2 dimensions. Let's then describe the system using the polar coordinates, such that: $x=r\cos(\theta)$ and $y=r\sin(\theta)$. Let's define two vectors, $\vec{r}$ and $\vec{\theta}$, such that:
$$\begin{cases} \vec{r}=x\vec{e}_x+y\vec{e}_y \\ \vec{\theta}=-y\vec{e}_x+x\vec{e}_y\end{cases},$$
where the vector $\vec{r}$ is such that is magnitude is the distance from the origin to the particle's position and $\vec{\theta}$ the vector whose magnitude is the angle between the positive half of the $xx$ axis and the vector $\vec{r}$.
Now, for commodity, let's define the vectors $\vec{e}_r$ and $\vec{e}_\theta$ the unit vectors that have the same direction of the vectors $\vec{r}$ and $\vec{\theta}$, respectively. Using this vectors I can write
$$\begin{cases} \vec{r}=r~\vec{e}_r \\ \vec{\theta}=\theta~ \vec{e}_\theta\end{cases}.$$
Comparing the above equation with the previous definition in terms of the Cartesian coordinates we have that $$\begin{cases}\vec{e}_r=\cos(\theta)\vec{e}_x+\sin(\theta)\vec{e}_y \\ \vec{e}_\theta=-\sin(\theta)\vec{e}_x+\cos(\theta)\vec{e}_y \end{cases}.$$
Because we'll need it a little further let's take the derivative with respect to time of $\vec{e}_r$ and $\vec{e}_\theta$. This is quite straight forward and we find:
$$\begin{cases}\frac{d\vec{e}_r}{dt}=\frac{d\theta}{dt}\vec{e}_\theta \\ \frac{d\vec{e}_\theta}{dt}=-\frac{d\theta}{dt}\vec{e}_r \end{cases}.$$
Finnaly we're in position to answer your question. Let's compute the expression for the velocity of the particle:
$$\vec{v}=\frac{d\vec{r}}{dt}=\frac{dr}{dt}\vec{e}_r+r\frac{d\theta}{dt}\vec{e}_\theta.$$
Let's now compute the acceleration:
$$\vec{a}=\frac{d^2\vec{r}}{dt^2}=\left[ \frac{d^2r}{dt^2}-r\left(\frac{d\theta}{dt}\right)^2 \right]\vec{e}_r+\left[ 2 \frac{dr}{dt}\frac{d\theta}{dt}+r\frac{d^2\theta}{dt^2} \right]\vec{e}_\theta.$$
In your specific problem the radius of the orbit doesn't change so the derivatives of $r$ are zero. We can further assume an idealized situation where the angular velocity of the particle doesn't change. So we're left with:
$$\vec{a}=-r\left(\frac{d\theta}{dt}\right)^2~\vec{e}_r.$$
And this is what we call the centripetal acceleration and, as you see, it always has the direction of $\vec{e}_r$ (in your case, horizontal).