A transfer matrix is a representation of a homogeneous linear function that approximates a behavior. As such, it is meant to work for all cases, at least approximately, over a range of inputs. So it doesn't depend at all on where the object and image are. In fact, you compute the image given the object position by using the unique, fixed transfer matrix characterizing an optical element. Before we begin, I believe your transfer matrix should be:
$$\left(\begin{array}{cc}1 &0 \\ -\frac{1}{f} & 1\end{array}\right)\tag{1}$$
instead of what you have written.
So, let's suppose a point source is at the focal point. At the source, we have a bundle of rays i.e. a set of state vectors of the form:
$$B(x)=\left\{\left.\left(\begin{array}{c}x_0\\\theta\end{array}\right)\right|\;\theta\in\mathbb{R}\right\}\tag{2}$$
meaning rays that pass through the transverse plane all at the same transverse position (namely $x_0$) all at different inclinations $\theta$ to the optical axis. After having propagated an axial distance $f$, the ray making an angle $\theta$ with the optical axis still has the same angle, but now its transverse position has changed to $x_0 + f\,\tan\theta\approx x_0 + f\,\theta$. Transfer matrices always describe a linearization of an optical system's behavior about the state of a chief ray, so the small angle approximation always holds. So now, our ray has the state:
$$\left(\begin{array}{c}x_0+f\,\theta\\\theta\end{array}\right)\tag{3}$$
so your task is to work out what the transfer matrix in (1) does to this state, i.e. simply to left multiply (3) by (1). In particular:
- What is particularly noteworthy about the angle of the ray after the transformation?
- Work out what this means physically.
and the answering of these questions should show you how the transfer matrix encodes the lens's collimating behavior for all points on the focal plane.