What lies between "perfectly smooth" and "perfectly rough"?

In elastic collisions of spheres, there are two approximations that can be made: Either they are perfectly smooth, and you ignore the rotation component, or they are perfectly rough and the rotation is such that there is no slipping.

Consider the situation of hitting a wall with a ball at a $$60^\circ$$ angle, not rotating. Its normal component of velocity will reverse no matter what. Its tangent velocity will remain the same if perfectly smooth, and it will decrease such that the point of contact is at rest at time of collision if perfectly rough (making the ball exit at more than $$60^\circ$$ from the wall).

If the ball is neither of these extremes, what is a good model? Does the percentage of speed that stays as "slip velocity" stay constant? Or does a portion of energy get converted? Can a greater normal component change behaviour in the tangent direction?

There is no answer that conserves energy. First, from elasticity, we know that the normal component of the velocity stays constant. We then have two unknowns: the new tangent velocity and the new rotation speed. Conservation of angular momentum around the point of contact (all forces act through it, so the net torque around it will be zero) provides one equation. If we use conservation of kinetic energy as the other equation, there are two solutions, one perfectly smooth, and the other perfectly rough, as demonstrated here in detail.

The more general model assumes two coefficients of restitution. A COR in the tangential direction $$c_T$$ determines roughness, while a COR in the normal direction $$c_N$$ determines the elasticity of a collision.

These coefficients are defined using the original and new velocities of the contact point. $$v$$ is the speed of the sphere's center of mass. $$\omega$$ is its rotation speed. Values after collision are represented by a hat. Tangential velocity is positive to the right, rotation is positive counterclockwise, $$r$$ is the sphere's radius:

$$c_N = \frac{\hat{v_N}}{v_N}$$ $$c_T = \frac{\hat{v_T} + \hat{\omega} r}{v_T + \omega r}$$

$$c_N = -1$$ is a perectly elastic collision, $$c_N = 0$$ a perfectly inelastic one. $$c_T = 1$$ is perfectly smooth, while $$c_T = -1$$ is perfectly rough.

The conservation of angular momentum dictates (the moment of inertia of the sphere being $$Jmr^2$$, $$J$$ being $$\frac{2}{5}$$ for a full sphere, $$\frac{2}{3}$$ for a hollow one, and $$1$$ for a hollow cylinder):

$$Jmr^2\hat{\omega} - mr\hat{v_T} = Jmr^2\omega - mrv_T$$

Combining this with the definition of $$c_T$$ and $$c_N$$, the results are:

$$\hat{\omega} = \frac{v_T(c_T-1) + r\omega(c_T+J)}{r(J+1)}$$ $$\hat{v_T} = \frac{JR\omega(c_T-1) + v_T(Jc_T+1)}{J+1}$$ $$\hat{v_N} = v_Nc_N$$

As noted here, $$c_T$$ may vary for the same sphere depending on the impact angle, and can be computed fro the coefficient of friction between the sphere and the wall. The more general problem of sphere-sphere collisions is studied here (starting around page 15 of the PDF).