given the position, velocity, mass and radius of the sphere/circles, how can I calculate the velocities after collision?
It would be best if it was in vector form not angle form.
given the position, velocity, mass and radius of the sphere/circles, how can I calculate the velocities after collision?
It would be best if it was in vector form not angle form.
$$ \boldsymbol{n} = \frac{ \boldsymbol{\rm pos}_2 - \boldsymbol{\rm pos}_1 }{ | \boldsymbol{\rm pos}_2 - \boldsymbol{\rm pos}_1 | } \tag{0}$$
Fist calculate the reduced mass of the system $$m_{\rm eff} = \frac{1}{ \frac{1}{m_1} + \frac{1}{m_2} } \tag{1}$$
Then calulate the impact speed $$ v_{\rm imp} = \boldsymbol{n} \cdot ( \boldsymbol{v}_1 - \boldsymbol{v}_2 ) \tag{2}$$
Here $\cdot$ is the vector dot product, yielding a scalar $v_{\rm imp}$ as the impact speed (not velocity).
Then calculate the impulse magnitude when the coefficient of restitution is $\epsilon$ $$ J = (1+\epsilon)\, m_{\rm eff}\; v_{\rm imp} \tag{3}$$
Find the change in velocity for each object as a result of the impusle $J$ along the contact normal $\boldsymbol{n}$
$$ \begin{aligned} \Delta \boldsymbol{v}_1 & = - \frac{J}{m_1} \boldsymbol{n} \\ \Delta \boldsymbol{v}_2 & = + \frac{J}{m_2} \boldsymbol{n} \\ \end{aligned} \tag{4} $$
The final velocities are just the before velocity vectors plus the change in velocity, $\boldsymbol{v}_i + \Delta \boldsymbol{v}_i$.
Confirm that the bounce back is a fraction of the impact speed $v_{\rm after} =-\epsilon v_{\rm imp}$
$$ \begin{aligned}\boldsymbol{n}\cdot\left(\left(\boldsymbol{v}_{1}+\Delta\boldsymbol{v}_{1}\right)-\left(\boldsymbol{v}_{2}+\Delta\boldsymbol{v}_{2}\right)\right) & =-\epsilon\,\boldsymbol{n}\cdot\left(\boldsymbol{v}_{1}-\boldsymbol{v}_{2}\right)\\ \boldsymbol{n}\cdot\left(\Delta\boldsymbol{v}_{1}-\Delta\boldsymbol{v}_{2}\right)+v_{{\rm imp}} & =-\epsilon\,v_{{\rm imp}}\\ \boldsymbol{n}\cdot\left(-\frac{J}{m_{1}}\boldsymbol{n}-\frac{J}{m_{2}}\boldsymbol{n}\right) & =-\left(1+\epsilon\right)\,v_{{\rm imp}}\\ \left(\boldsymbol{n}\cdot\boldsymbol{n}\right)\left(\frac{1}{m_{1}}+\frac{1}{m_{2}}\right)J & =\left(1+\epsilon\right)\,v_{{\rm imp}}\\ \left(\frac{1}{m_{1}}+\frac{1}{m_{2}}\right)\left(1+\epsilon\right)\frac{1}{\frac{1}{m_{1}}+\frac{1}{m_{2}}}\,v_{{\rm imp}} & =\left(1+\epsilon\right)\,v_{{\rm imp}}\\ \left(1+\epsilon\right)\,v_{{\rm imp}} & \overset{\checkmark}{=}\left(1+\epsilon\right)\,v_{{\rm imp}} \end{aligned} $$
The problem has a two-step solution. Vector formulae are valid for any space dimension. Therefore I'll speak about spheres, but the same applies to circles (or Hyperspheres in n-dimensional spaces).
Step 1 may be trivial (if there is no force but the contact force at collisions) or more complex, if, in addition to the hard repulsion at contact, there is also a continuous force field between the spheres. Let's assume that there is no continuous force and the spheres move at a constant velocity between collisions. in such a case, the time up the collision of one sphere of radius $R_1$ that is at ${\bf r}^{(0)}_1$ at $t=0$ with velocity ${\bf v}_1$ and a second sphere of radius $R_2$ at ${\bf r}^{(0)}_2$ at $t=0$ with velocity ${\bf v}_2$, is given by the positive real solution (if it exists) of the algebraic second-degree equation resulting from the condition that at a time $t>0$ the centers of the two spheres will be at a distance $R_1+R_2$. In terms of the positions at the time $t$: $$ {\bf r}_i(t) = {\bf r}^{(0)}_i+{\bf v}_it $$ the condition is $$ |{\bf r}_{12}(t)|=|{\bf r}_1(t)-{\bf r}_2(t)|=R_1+R_2. $$ By introducing the relative velocity ${\bf v}_{12}={\bf v}_1-{\bf v}_2$, the equation becomes $$ |{\bf v}_{12}|^2t^2+2\left( {\bf r}_{12}^{(0)} \cdot {\bf v}_{12}\right) t + |{\bf r}_{12}^{(0)}|^2-(R_1+R_2)^2=0. $$
Step 2. To find the relation between velocities after the collision (${\bf v}'_i$), and before the collision (${\bf v}_i$), it is convenient to express the collision conditions in the center of the mass frame and then to transform back to the laboratory frame. In the center of the mass frame, the elastic conditions correspond to the equality of the kinetic energy before and after the collision, the momenta of the two spheres before and after the collision are opposite ($m_1 {\bf v}'_1=-m_2{\bf v}'_2$, and $m_1 {\bf v}_1=-m_2{\bf v}_2$).
The key condition for determining the velocities after the collision is that the component of each particle momentum parallel to the tangent plane to the contact between the two spheres remains the same after the collision, while the perpendicular component is reversed. Taking into account that the unit vector ${\bf \hat r}_{12}=\frac{{\bf r}_{12}}{|{\bf r}_{12}|}$ is perpendicular to the tangent plane at the contact point, it is a simple exercise to obtain the formulae for the final velocities: $$ {\bf v}'_i={\bf v}_i - \frac{2m_j}{m_i+m_j}\left( {\bf \hat r}_{ij} \cdot {\bf v}_{ij}\right){\bf \hat r}_{ij} $$