This is going to be a bit of a half baked answer but here it is.
The usual picture for light with orbital angular momentum is light in a Laguerre-Gaussian Mode mode with non-zero azimuthal index. This sort of field has phase wrapping azimuthally around the propagation axis.
We suppose without loss of generality that the atom is on the propagation axis of the beam*
As I point out in the linked post and as you emphasize in the question, typically when we think about interactions between atoms and light we work in the dipole approximation meaning we assume the electric field is homogeneous across the whole volume of the atom. In the case of light with angular momentum we can see this is not the case. Consider the simplest Laguerre-Gaussian mode with a non-zero phase wrapping.

If the atom is in the center of this picture we can clearly see that the dipole approximation is NOT satisfied. This is because the phase of light on the right half of the atom is opposite to the on the left side of the atom. This means no matter what we do with, for example, the scaling of the beam, the atom will never be experiencing a homogeneous field. This means the dipole approximation fails.
When the dipole approximation fails all of the "usual" selection rules that you learn in intro classes are thrown out the window and you need to use higher order selection rules.
I don't know enough about all the details of these selections rules but you can see that this pattern could impart angular momentum to the electron around the nucleus.
And that is my answer to your question. Just like how spin angular momentum of light imparts angular momentum to the electron (atomic "internal" angular momentum) orbital angular momentum of light also imparts momentum to the angular momentum of the electron.
Note that you state
it is clear that since atoms are so small (<0.1 nm), they don't really "feel" the OAM of light.
And you drew this conclusion from my answer to the other question. This conclusion is incorrect. It is possible to have optical fields which vary on atomic length scales (Like the one I have shown above). It just requires using tightly focused or very high order optical fields which need to be carefully aligned so that their regions of large gradients are overlapped with the atom. For example, if the Laguerre-Gaussian mode depicted above is made to be a large beam (waist of a few mm for example) and is offset so that the atom is like in the middle of the red part then the field will look homogeneous to the atom. It is only when the atom is at the central vortex that it strongly experiences the field inhomogeneity.
All of that said you may be interested in Spin-orbital-angular-momentum coupled Bose-Einstein condensates. Note that here the light is not coupled to a single atom but rather to a quantum gas of many atoms in the BEC state. In this case the orbital angular momentum of the light is transferred into the orbital angular momentum of the BEC.
*We can always choose a basis for the optical field which is centered on the atom. If beam is displaced from the atom then in this particular basis it will be composed of a large number of high order modes.