Horrocks did not, in fact directly use his Venus transit observations to measure the AU, but to confirm a relationship that he postulated between planetary radii and orbital distance, using which the AU could calculated simply from an estimate of the radius of Earth.
During the early 17th century it was widely believed that the size of a planet was related in some way to its distance from the Sun. As expressed by Kepler (1618):
Nothing is more in concord with nature than that the order of
magnitude should be the the same as the order of the spheres, so that
among the six primary planets, Mercury should ... obtain the most
narrow sphere; that next to Mercury should be Venus, which is larger,
but still smaller than Earth's ...
The significance of Horrocks's (1639) Venus transit observation, combined with earlier observations of Mercury and the contemporary estimates of the relative sizes of the orbits of the Mercury, Venus and Earth, is that it allowed him to be more explicit1:
All planets (with the exception of Mars) are the same angular size
when seen from the Sun, this size being 28 seconds of arc.
From this "law"2 (sometimes called "Horrocks Law") arriving at an estimate of the AU is a matter of simple trigonometry, involving only the Earth's radius,3 $R_E$, since if the Earth subtends an angle of $28''$ when viewed from the Sun
$$R_E / 1 \text{AU} = \tan (28''/2)$$
so that
$$1 \text{AU} = (\tan 14'')^{-1}R_E \approx 14,733 R_E$$
(1) Horricks observed an angular size for Venus of $76''$ and used contemporary estimates of the relative Sun-Venus and Sun-Earth distances, taking orbital eccentricities into account, to compute the angular size of Venus as seen from the Sun:
$$76'' \cdot ( 0.98409 \text{AU} - 0.7200 \text{AU}) / 0.7200 \text{AU} \approx 27.876''$$
A similar calculation based on Gassendi's 1632 observation of Mercury (using $20''$ and Kepler's values for Sun-Mecury and Sun-Earth distances) also produced an angular size of about $28''$. This, along with the general beliefs of the time, were apparently enough to convince Horrocks that the angular size from the Sun was $28''$ for all planets — notably Earth, which made his calculation of the AU in terms of $R_E$ possible.
(2) The term is modern. Horricks was more careful: "I do not put forward this conjecture as a certain demonstration, but as a probability".
(3) For which many contemporary estimates were available.