This is caused by chromatic aberration in the eye: short waves (blue) are refracted more (by the cornea and the crystalline lens) than long waves (red light). This isn't visible in normal conditions, but can be observed using your method. A good overview of the eye can be found in this ref., see page 49 for chromatic aberration.
Because of it, a multi-color scene can never be perfectly focused on the retina: a point source in focus for green (550 nm) will be myopic (focal point in front of the retina) for blue (480 nm) and hyperopic (focal point behind the retina) for red (656 nm). The image on the retina is a green point, and a red and blue blurred region around that point (the reality is more complex, see ref. page 34 for spot diagram).
In the figure above, green is in focus, red and blue aren't (all the rays drawn in the picture come from the same point in the distance). The red component will hit the retina to the left of the focal point when coming through the left half of the lens, and to the right when passing through the right half. The distance from the focal point is proportional to the distance from the center of the lens, light passing through the center will hit the focal point regardless of wave length.
When you line up your finger to the right of the edge, as in your second diagram, you are covering most of the cornea, and light coming from the observed edge can only pass through the left outer region of the iris. On the retina, the sky and clouds are on the right, the black area is on the left:
Assuming the edge is in focus in the green wavelength (which component the eye uses to focus depends on the distance), the corresponding red component will be a blurred line to the left of it, and a blurred blue one to the right. Instead of a sharp transition from black to white, you get a diffuse transition from black to red, to yellow (red + green), to white (red + green + blue).
Based on this reasoning, you can also deduce that an (imperceptible) magenta edge is present when your field of vision is unobstructed (ie when you remove your finger): The red component will still be there, and the light passing through the right side of the lens adds a left shifted blue component. But in that situation, the white edge is much brighter, obscuring the effect.
For example, with your finger at 2 cm of your eye, the edge at 2 m distance, a pupil size of 3 mm and 1 cm of the black region visible, you can calculate (using the formulas for a circular segment with R = 1.5 mm and h = 0.1 mm) that light coming from the edge only reaches 1.02% of the total surface of the pupil. Without the finger blocking the light, the edge will be 100 times brighter, making the magenta line next to it impossible to see, in the same way a (non-transparent) narrow object in front of a bright lamp is perceived as dark or black, regardless of its true color (due to the limited threshold sensitivity dL/L, see ref page 11, and lateral inhibition in the retina).
The magnitude of the aberration is about -1.5 dpt for blue and +0.5 for red.
In theory, correcting it would improve visual acuity in white light, and this has been confirmed in lab tests, but it is difficult to implement in glasses or contact lenses (the corrective lenses used in the lab required accurate alignment and restricted the field of vision). The human eye partly compensates for the effect: the fovea, the area responsible for sharp central vision, is less sensitive for blue light (see page 10 of ref.).
(The first image is an adaptation of "Chromatic aberration lens diagram.svg", DrBob, English language Wikipedia)