Basically is the black hole itself affected by the matter going into it? Does it either get pulled by gravity toward things or have its own velocity affected by matter that "impacts" it?
As long as you stay well outside the event horizon a black hole behaves just like any other massive body. For example the first black hole identified, Cygnus X-1, is part of a binary system. It and its partner both orbit their common centre of mass. So it is an example of the trajectory of a black hole being affected by the gravitational field of another object.
Response to comment:
The system of a black hole and another mass conserves momentum (linear and angular, as Anuar says in his comment) just like any system of masses. That means the momentum after the collision is the same as the momentum before the collision. If you fire a body with momentum $p$ at a black hole then after the collision the momentum of the (now slightly larger) black hole will have changed by $p$.
Likewise with angular momentum. To return to Cygnus X-1, because the black hole and companion star orbit each other the system has some angular momentum $L$. After the two have merged (which must happen eventually) the angular momentum of the (now substantially larger) black hole will have increased to conserve the total angular momentum.