What local forces cause the damage? Tidal forces. Tidal forces can only be neglected in extremely small regions, and you have an extended body. More details follow.
You suppose an extended body, bound together, interacting gravitationally with a black hole. With the center of mass staying outside the photon sphere, moving on a more than barely unbounded orbit. But a large enough extended body that part of it crosses the event horizon, but you want the Roche limit to be smaller than the event horizon.
I can see why people criticized the original question. If you define the Roche limit to be the distance at which a satellite breaks up from tidal forces, then it appears to be a contradiction on the face of it.
But I'll start with a warning about center of mass motions. When you are losing matter, the center of mass can move faster than light. Really. Imagine a very long train, moving at a speed $v$, but the link halfway through breaks (so the back half slows and stops) and then the center of mass of the moving train jumps to where the first 25th percentile (1st quartile) originally was. Then the link halfway through the remaining train breaks. So the new center of mass is where the original first 12.5th percentile was. This could happen very quickly. And the center of mass can thus jump a large distance in a short amount of time. So center of mass motions are not at all like real motions of real particles. That was a warning because it seemed like you didn't know this (and this is true even in Newtonian Physics), but this does mean you might be able to start within the photon sphere and still have part of your star escape since the center of mass can move faster than light if you are willing to leave something behind. Don't get too excited about that, because it is deep or fantastic, but it can avoid painting yourself unnecessarily into a corner by falsely thinking center of masses have to do things they don't have to do.
But you asked about the case where the center of mass stays out of the photon sphere. So lets do that. So now the question remains about whether the Roche limit can be inside the event horizon. If your center of mass is outside the photon sphere you are at areal coordinate 1.5 $R_S$ (where $R_S$ denotes the Schwarzschild "radius") or farther. You want the actual radius to be large enough to have the center stay outside the photon sphere yet have part go into the event horizon. That might look like you need a radius of 0.5 $R_S$ or more, but the distance between areal coordinate 1.5 $R_S$ and areal coordinate 1.0 $R_S$ is actually more than 0.5 $R_S$. Even if you haven't studied GR you can see that by looking at one of those funnel pictures and note that the areal coordinates is the circumference divided by $2\pi$ (by definition, you can also take the surface area and divide it by $4\pi$ and then take the square root), and since the funnel extends in a third direction it requires more distance travelled to get to larger or smaller areal coordinate than you'd expect if the areal coordinate were just distance from the center as it is in uncurved spacetimes. But since that was the same factor for the radius of the star and the radius of the black hole and they need to be about 50% of the size with each other to straddle the region from event horizon to photon sphere, it might not be anything to worry about.
But that scale, that the satellite is maybe near to 50% the size of the black hole. That itself does pose serious problem to the normal expression for the Roche limit. The normal expression assumes that the radius of satellite is much smaller than the distance between the centers. This is most definitely not going to hold in this case.
So normally you consider the tidal force as the difference in two forces, say from the center and edge. So if the centers are a distance $d$ apart, and the satellite has a radius $r$ then the difference of the forces on a small mass $u$ nearest the black hole is
Where the approximation only holds when $r << d$, which doesn't hold in our case. At all. Let's try to look at our situation. In our situation the mass closest to the black hole is actually inside the event horizon. Now in GR gravity isn't a force, but we know that any force applied to the part inside will fail to keep it at rest, so there is a sense where that force is infinite. So it is reasonable to only look at tidal forces outside the horizon.
So let's look at near misses, where the passing star has parts graze the event horizon as close as we want. If that is enough to rip parts off, then we know the tidal forces are strong enough. We know that it is possible to escape the black hole if you are outside the horizon. But if the only way to fight out of the black hole is the binding of the satellite, then we can take that as fixed and see how close we have to graze the event horizon before we leave parts behind.
If the whole satellite stays outside of the photon sphere there should be no problem, and this is far from grazing. Contrary to popular mythology, the satellite is an extended body so can't just move on a background spacetime distorted by the blackhole alone, but instead the satellite really distorts spacetime itself as well. But that is hard to do correctly and you said you didn't know much GR so let's ignore that.
Once we ignore that we can use a trick. Imagine a series of stars with the same center of mass but larger radii. The ones that have a radius extending into the black hole lose part of their outer layer. That happens to all of them, regardless of how little they went over and how strongly they were bound together. And it was tidal forces that pulled them in regardless of the binding. So the cohesive forces relative to the far away center of the star weren't enough to keep the star parts on the black hole side of the event horizon attached, and the forces vary continuously so they must get arbitrarily large. Now these forces are a relative force, they are between two regions, a differential force. For a nonextended body they would vanish. But tidal forces are real. The geometric version would be that say a blob of water minimizing its surface area finds less surface area if it bulges towards and away from a star. An extended body notices that spacetime is curved differently in different regions and it's mutual interactions act differently because of that than they do in flat space. So in a sense the star is tearing itself apart because it's ability to accommodate the different geometry on the different sides.
It is also important the tidal forces disappear not just for small regions of space, but only for small intervals of time too. That changing geometry can also be a changing from time to time. And the time part of spacetime can and is curved too. In fact for everyday situations that is the important effect.
So the tidal tearing apart doesn't happen right away. So the first part of the star that dips in doesn't notice much at first, only really small tidal effects in that really small region for that short moment it crosses. But those small differences build up because each part is just a bit off from their neighbor and the star becomes stretched out. It's as the star pulls away the different parts are ageing differently which affects how they cohere together and propagate, so the outer parts pull away. If you go back to the water blob that found less surface area if it bulges towards and away from the black hole you see that it pulls away more than normal and as the center of mass tracks it is harder to drag those parts with it. All tidal forces, and yet also it is all also the star ripping itself apart.
OK. So we could do that again and again for smaller and smaller stars eventually we have a star that just barely has a part on the inside inside, and it is still torn apart. but now if you removed that outer layer the temperature and pressure of the remaining star wouldn't be that different (since the outer layer is so thin). And without that outer layer there is actually less pressure from that now gone outer layer holding that now outermost layer towards the remaining star, and the larger star was torn apart even when/if it had a fictional stronger (say nongravitioanl) binding. So it's bound less tight and would have been stretched out even if bound more tightly. It gets torn apart. Technically you have to be more careful when computing that last argument. You can find out how much that now never existing layer pushed down, and choose a small region of space and time right when it was getting super close to the event horizon and argue that the lack of a push now means we know how it moves. And by choosing a frame localized in time and space that is falling towards the black hole, we have that it only failed to cross when it had an extra push away from the event horizon and now does not have that extra push. It's more rigorous if you select actual masses and radii, but it doesn't matter for the final result. And it's technically not right at all if we don't let the star curve spacetime too and instead have it exert gravitational forces on each other. But if the satellite is a star and not a black hole or neutron star itself that might be fairly accurate.
I'm not sure why you implicitly assumed the black hole to be larger than the star. If you had a star sitting there minding it's own business and a very small black hole whizzed by but stayed a few Schwarzschild radii away from the center of the star it would just thread through the star and remove a small tube of matter and keep going. Obviously the center of mass of the star stayed out of the photon sphere, and part of it entered the event horizon and got eaten. And it was tidal forces. The local pull of the tiny black hole pulled in some matter because locally it was stronger than the pull of the further interior portions of the star.