Here is a purely geometrical way to think about this
Edward says it is possible to cut a wedge out of a flat spacetime and glue the edges together. So in my mind this looks like a paper cone.
A cone is flat precisely because it can be created by rolling up a flat sheet. Rolling up preserves the metric on the interior of the sheet (not on the boundaries where the wedge was cut out). Changing the metric requires stretching or topological changes. Rolling up does not involve stretching. Therefore the intrinsic geometry of the cone is defined to be the geometry of the flat sheet except at the seam. Since a cone has rotational symmetry this implies it is flat everywhere except the tip. The tip is preserved by rotations about the axis, so it is not equivalent to any other point where the geometry is known to be flat.
It immediately follows that parallel transport on the cone is given parallel parallel transport on the flat sheet, except at the seam, since parallel transport is defined in terms of the metric (the Levi-Civita connection). Transport across the seam is transport across the missing wedge of the flat sheet. To preserve continuity when transporting across the missing wedge, it is necessary to perform a rotation by an angle equal to that of the missing wedge. To see why think about transporting basis vectors of the polar coordinate system across the seam.
Parallel transport around the tip requires crossing the seam and, therefore, applying the above rotation. That rotation is the angle deficit of the path. The angle is independent of the path since it is determined by the missing wedge. In particular it is the same for arbitrarily small paths around the tip. Therefore all the curvature is concentrated in the tip.
Video: Leonard Susskind Leonard Susskinddemonstrating this demonstrating this with an actual sheet of paper