A bound elliptical orbit around a Schwarzschild black hole can get arbitrarily close to $r = 2 r_s$ (where $r_s = 2 M$ is the Schwarzschild radius) but no closer.  Deriving this result is a good exercise for students learning about the Schwarzschild geometry, so I won't go through all the details, but the basic sketch of the proof is as follows:

 * Recall that a massive particle moving in a Schwarzschild geometry is equivalent to a particle moving in a classical "effective potential" given by
$$
V_\text{eff}(r) = - \frac{M}{r} + \frac{\ell^2}{2 r^2} - \frac{M\ell^2}{r^3},
$$
where $M$ is the mass of the black hole and $\ell$ is the specific angular momentum of the particle.
 * Note that for a bound orbit, we must have $V_\text{eff}(r) < 0$ at all times.
 * Find the points at which $V_\text{eff}(r) = 0$ for a given value of $\ell$.  This will be the closest possible value of perihelion for a bound orbit for a particular value of $\ell$.
 * Find the value of $\ell$ that allows for the closest perihelion.  It turns out to be $\ell = 4M$, and for that value of the angular momentum you must have $r > 2 r_s$ to satisfy $V_\text{eff}(r) < 0$.