The basic meaning of the latin word "momentum" is movement or the power to move. The specific use of the term to mean mass $\times$ velocity was a comparatively late (seventeenth century) development.
But "moment", the anglicised version of the word, had already gone into speech to mean the power to get things going in the sense of importance or consequence; Shakespeare's Hamlet talks about "enterprises of great pith and moment". The word is still occasionally used in this sense in everyday speech, and we have the adjective "momentous".
So the turning moment of a force is the force's ability to turn things. Multiplying the force by the perpendicular distance from a point is what measures the force's ability to do this, its consequence, indeed its moment!
I suspect that using the term "moment" when we multiply of a quantity by a significant distance, as when we calculate the electric dipole moment of a collection of charges as $\sum Q \vec r$, is modelled on the calculation of the moment of a force (or the combined moment of a number of forces). Indeed 'moment' has pretty much taken on the meaning of a quantity multiplied by in some way by a distance or displacement from an axis or from a point, but the basic idea of giving importance to the quantity (in a particular context) by so doing is still in the background.
And what about the moment of inertia of a rigid body? To calculate this we multiply each mass element by a distance squared, rather than by a distance, before summing. But again we are constructing from the masses a quantity of moment when it comes to rotation. To be more specific, multiplying by $r^2$ gives us what is now called the second moment of mass, the first moment being multiplication of each mass element by its distance from an axis before summing (and quite likely dividing by the total mass in order to find the position of the centre of mass). The zeroth moment, $\sum m r^0$, is the simple sum of masses, but this is no doubt a later extrapolation from the original terminology.
This answer probably seems rather hand-wavy, but don't forget that it is only names that we are talking about. We could call the moment of inertia "Charlie" and it wouldn't affect the Physics. Names of quantities in Physics are usually very logical and helpful, but not always, as in the case of electromotive force. Moment is not quite in this category, but it's possibly not the most transparent of terms.