Mechanics of colliding irregular shapes in 2D or 3D I am familiar with the mechanics of colliding spheres but I am trying to understand how this generalizes to irregular shapes (eg. 2 cubes colliding in 3D space). Clearly this is a more involved problem since:


*

*There are many ways that the objects can come into contact (face-face, edge-face, edge-edge, vertex-face, etc).

*The rotation of the object must be considered as it is not spherically symmetrical.

*The centers of mass of the colliding objects will not necessarily be on the line of impact, leading to torque forces.
I understand how to calculate the result of something like a rigid square in a 2D space colliding with a fixed, infinite surface. However, when the surface is instead replaced with another polygon, the problem becomes more difficult.
It also occurs to me that there is a disconnect between the real world and any mathematical system which models perfect geometric shapes (with infinitesimal vertices) which do not really exist in the physical world. Therefore, even when it appears as though a sharp point is colliding with a surface, in the real world there is always a finite contact area. So how does this affect the problem of modelling general collisions?
To be clear, I am not looking for information about collision detection. Actually I was quite frustrated that I was unable to find any material on this subject with a quick google search... One might have expected this to be a standard problem in undergraduate mechanics and yet all I can find online only deals with the collision of spheres.
Is it simply that a problem of this nature is too complex to be explained in terms of the Newtonian formulation of classical mechanics? Would it be a sensible approach to use Lagrangian/Hamiltonian mechanics instead?
If anyone can either give me some insight into how to calculate the result of such collisions, or direct me to some reading material elsewhere, that would be greatly appreciated.
Edit: One may suppose for simplicity/illustration that I would like to model the collision of exactly 2 hard cubes in 3D space that are known to collide at some time but for which the exact point of impact and the velocities/rotation of the cubes could be anything.
 A: No, the problem is not too complex for Newtonian Mechanics. Newtonian equations of motion for a collision between 2 bodies are relatively easy to write down and code. The Lagrangian/Hamiltonian approach is better suited to constrained motion of connected bodies.
The same principles apply in 3D as in 2D : conservation of linear and angular momentum, and possibly also of kinetic energy. The 3D problem is considerably more complex for the reasons you give. In 2D each rigid object has 3 degrees of freedom (2 translational, 1 rotational), in 3D there are 6 degrees of freedom (3 translational, 3 rotational). In 2D there are 3 types of collision between edges and corners (E-E, C-E, C-E) while in 3D there are 6 types between faces, edges and corners (F-F, E-E, C-C, E-F, C-F, C-E). The same complexities arise whatever mathematical formulation you use. 
You are right about the disconnect between mathematical models and reality. Mathematical edges, corners and points make it ambiguous in which direction the contact forces act during a collision. How you resolve this dilemma is up to you. On a small enough scale all real corners and edges can be defined as continuous surfaces, then all collisions have a definite plane of contact. This comes at the expense of complexity in the model. If accuracy is not required, only a realistic "feel" (eg in computer gaming), then a "rule of thumb" can be applied to a less complex model. 
There is also the rigid body model itself, with no deformations, which implies that collisions are instantaneous. This excludes the real possibility of multiple-body collisions, even in 1D. See Is my method here for solving this 1D 3-body collision problem correct? and Multiple colliding balls. However, although the rigid body model is unrealistic (by prohibiting multiple collisions) it does make coding simpler because the outcome of each collision is easily determined. 
Collsion detection gets a lot more attention than collision response because it is used a lot more often, it requires a complex decision and an efficient algorithm. When a collision is detected a straightforward set of calculations can be followed to find the outcome. The edges/faces at which the collision occurs is already known from the collision detection algorithm. 
However, there are some online tutorials and documents available for calculating the effect of the collision. For example :
MyPhysicsLab
Euclidean Space
Chris Hecker on Rigid Body Dynamics
Micheal Manzke lecture ff slides 23 onwards
Randy Gaul tutorial
Carnegie-Mellon University  : Rigid Body Simulation notes Part I and Part II
Useful questions on Physics Stack Exchange include :
How multiple objects in contact are resolved in an inelastic collision, when edge normals don't "line up"
Calculate force between rotating objects
What is the initial angular momentum of a rigid body given an offset impulsed force?
Off-center impulse equations 

As a start I recommend that you extend your 2D model to handle all collisions between 2 rectangles, including corner-edge and corner-corner collisions. This will require you to decide how to model corners. When you have a working 2D application you can upgrade it more easily to 3D, instead of trying to code a 3D application from scratch. 
