I have not watched the video but...
The physical differences in classical reality are as important as the mathematical similarities.
The curvature of space-time is NOT a force field, as in Electromagnetism. The Christoffel symbol, or connection coefficient, creates force on test particles in the classical theory. This has always been an issue and one of the things that lead first attempts at unification to add fields to the metric. Coincidentally the extra connection terms transform as a curvature tensor.
Quantum mechanics is, for better or worse, something we do to a classical system (with the possible exception of non-relativistic spin, but in the Dirac theory and in QFT the field is seen as a classical structure on the manifold). There are more differences to discuss but I think this is the biggest. The very nature of the force is different at a classical (and empirical) level.
As for EM being Abelian that is due to the nature of the gauge symmetry, U(1) is an Abelian group, SU(2), SU(3), etc are non-abelean groups. Just because the curvature is a tensor doesn't mean it isn't Lie Algebra Valued (it is). It carries 2 space-time indices and two lie algebra indices. For U(1) the internal indices are trivial. The group action is on an "internal degree of freedom" like charge, phase, iso-spin, etc. That feature does not get in the way of seeing the similarity. In GR the "internal indices" are space-time directions or vielbien directions in the local tangent space at each point in the manifold.
To unify GR and SM researchers have been trying to capitalize on any similarity they can find (and this is a smart move, it's not made up) but to really accept these one needs to change the way we look at F = ma in GR.
An other difference is that the field action is first order in curvature while gauge theories are second order, again for classical theory first order R --> second order Connection, which is the force field. A higher order GR theory would make things look more alike but produce new issues.
The curvature in GR produces a force gradient dF/dx, which relates to perturbations in orbital motion of test particles.