The two bodies exchange (scalar) momentum $p$ along the contact normal $\boldsymbol{n}$, and through the contact point $\boldsymbol{r}$ (relative to the center of mass).
Here is a schematic of the situation
Linear momentum is inherently balanced because whatever momentum vector $p\, \boldsymbol{n}$ is added to (2) it is subtracted from (1).
$$ m_1 \boldsymbol{v}_1 + m_2 \boldsymbol{v}_2 = \left( m_1 \boldsymbol{v}_1 -p \boldsymbol{n} \right) + \left(m_2 \boldsymbol{v}_2 + p \boldsymbol{n} \right) $$
$$ m_1 \boldsymbol{v}_1 + m_2 \boldsymbol{v}_2 = m_1 \boldsymbol{v}_1^\star + m_2 \boldsymbol{v}_2^\star $$
where $\square^\star$ are the values after the impact.
Similar argument can be made for angular momentum, as the vector $\boldsymbol{r} \times p\, \boldsymbol{n}$ is added to (2) and subtracted from (1).
$$ \boldsymbol{r} \times m_1 \boldsymbol{v}_1 + \mathbf{J}_2 \boldsymbol{\omega}_2 = \left( \boldsymbol{r}\times m_1 \boldsymbol{v}_1 - \boldsymbol{r} \times p\,\boldsymbol{n} \right) + \left( \mathbf{J}_2 \boldsymbol{\omega} + \boldsymbol{r}\times p\,\boldsymbol{n} \right) $$
$$ \boldsymbol{r} \times m_1 \boldsymbol{v}_1 + \mathbf{J}_2 \boldsymbol{\omega}_2 = \boldsymbol{r} \times m \boldsymbol{v}_1^\star + \boldsymbol{r} \times \mathbf{J}_2 \boldsymbol{\omega}_2^\star $$
The problem boils down to finding the one scalar momentum exchange value $p$ (usually called the impulse). This is accomplished by the law of contacts, relating the bounce relative speed to the impact relative speed. Notice I said speed and not velocity, as only the velocity along the normal $\boldsymbol{n}$ is important. The law states that $v_{\rm bounce} =-\epsilon\, v_{\rm impact}$, where $\epsilon$ is the coefficient of restitution.
Here velocities are transformed from the center of mass to the contact point and projected along the contact normal.
$$ \begin{aligned} v_{\rm impact} & = \boldsymbol{n} \cdot \left( \boldsymbol{v}_1 - \boldsymbol{v}_2 \right) \\ v_{\rm bounce} & = \boldsymbol{n} \cdot \left( \boldsymbol{v}_1^\star - \boldsymbol{v}_2^\star \right) \end{aligned}$$$$ \begin{aligned} v_{\rm impact} & = \boldsymbol{n} \cdot \left( \boldsymbol{v}_1 - \boldsymbol{v}_2 + \boldsymbol{r} \times \boldsymbol{\omega}_2 \right) \\ v_{\rm bounce} & = \boldsymbol{n} \cdot \left( \boldsymbol{v}_1^\star - \boldsymbol{v}_2^\star + \boldsymbol{r} \times \boldsymbol{\omega}_2^\star \right) \end{aligned}$$
and from the conservation of momentum you have
$$ \left. \begin{aligned} m_1 \boldsymbol{v}_1^\star & = m_1 \boldsymbol{v}_1 - p\,\boldsymbol{n} \\ m_2 \boldsymbol{v}_2^\star & = m_2 \boldsymbol{v}_2 + p\,\boldsymbol{n} \\ \mathbf{J}_2 \boldsymbol{\omega}_2^\star &= \mathbf{J}_2 \boldsymbol{\omega} + (\boldsymbol{r} \times \boldsymbol{n}) p \end{aligned} \; \right\} \; \begin{aligned} \boldsymbol{v}_1^\star & = \boldsymbol{v}_1 - \tfrac{1}{m_1} p \boldsymbol{n} \\ \boldsymbol{v}_2^\star & = \boldsymbol{v}_2 + \tfrac{1}{m_2} p \boldsymbol{n} \\ \boldsymbol{\omega}_2^\star &= \boldsymbol{\omega}_2 + \mathbf{J}_2^{-1} ( \boldsymbol{r} \times \boldsymbol{n} )p \end{aligned} $$
The final equation compiled from the above and to be solved for $p$ is
$$ \boldsymbol{n} \cdot \left( - \tfrac{1}{m_1} \boldsymbol{n} - \tfrac{1}{m_2} \boldsymbol{n} + \boldsymbol{r}\times \mathbf{J}_2^{-1} ( \boldsymbol{r} \times \boldsymbol{n} ) \right) p = -(1+\epsilon)\, \boldsymbol{n} \cdot \left( \boldsymbol{v}_1 - \boldsymbol{v}_2 + \boldsymbol{r}\times \boldsymbol{\omega}_2 \right) $$
$$ \boxed{ p = \frac{(1+\epsilon)\, \boldsymbol{n} \cdot \left( \boldsymbol{v}_1 - \boldsymbol{v}_2 + \boldsymbol{r}\times \boldsymbol{\omega}_2 \right)}{ \tfrac{1}{m_1} + \tfrac{1}{m_2} -\boldsymbol{n} \cdot \left( \boldsymbol{r}\times \mathbf{J}_2^{-1} ( \boldsymbol{r} \times \boldsymbol{n} ) \right)} } $$
Now back substitute $p$ in the expressions above for $\boldsymbol{v}_1^\star$, $\boldsymbol{v}_2^\star$ and $\boldsymbol{\omega}^\star$.
Note that $\cdot$ is the vector dot product, and $\times$ the vector cross product. Also note that $\boldsymbol{n}\cdot\boldsymbol{n}=1$.