I have been studying general relativity by B. Schutz. It says locally straight lines on a sphere does not remain parallel, as great circles do intersect on a sphere.
Below is a picture where two cirlces, $C^1$ (great circle) and $C^2$ (not a great circle) do remain parallel if we extend it to the entire sphere, $S^2$. So why the author claimed that locally straight lines usually do not remain parallel on curved space.
Your curve $C_2$ is not a straight line.
There is an easy way to see that $C_2$ isn't a straight line without having to resort to differential geometry. A straight line is the shortest distance between two points$^1$, so consider two points on $C_2$ on opposite sides of the pole:
I've drawn $C_2$ as the red curve on a perspective diagram and also in the side on view. We'll take the sphere to have unit radius for convenience.
$C_2$ is a circle of radius $r = \sin\theta$ so the distance between opposite points measured along the circle is just half the circumference:
$$ s_1 = \tfrac{1}{2}2\pi r = \pi\sin\theta $$
Compare this with the distance between the points if we follow the line through the pole shown on the right diagram by the dashed line above the sphere. This is a segment of a great circle so the length of the segment is:
$$ s_2 = 2\theta $$
And for $0 \le \theta \le \pi/2$ the distance $s_2$ is always smaller than $s_1$. That means the curve $C_2$ cannot be a straight line because it isn't the shortest distance between the points. Rather than go though the algebra to show that $\pi\sin\theta \ge 2\theta$ I simply drew a graph to show this:
Note that the two distances are equal only for the trivial case where $C_2$ has zero radius and $\theta = \pi/2$ i.e. when $C_2$ is a great circle.
$^1$ not strictly true. A straight line extremises the path length, but let's overlook this to keep the argument simple.
It's easy to get confused about locally straight lines on a sphere, especially when you do it by eye as you have.
Instead, a method should be adopted. Now, the straight line on a plane is simply the shortest distance between two points. We can draw this easily enough by eye, but another method is to stretch a string between the two points and pull it taut.
If we now do this on the sphere you'll find the great circles and these are different from the circles that you have drawn on the sphere. You should also see quite quickly that any two will always intersect. So the geometry is not Euclidean.
