I don't know whether there are any general formulas concerning engine braking, but you could incorporate drag proportional to the car's velocity:
$$ F =ma= -bv $$
where $b$ is a constant and $v$ the car's velocity. This should reproduce a similar effect to engine braking. The deceleration $a$ is then also proportional to the car's velocity:
$$ a = -\frac{bv}{m} $$
with $m$ the car's mass.
Edit:
Real drag due to aerodynamics is usually taken to be proportional to $v^2$, and also the (frontal) surface area of the car:
$$ F\sim\rho A v^2 $$
where $\rho$ is the density of the medium (in this case: air). I think that the model proportional to $v$ would emulate engine braking slightly better, but you can try both and see which feels more natural.