To be clear, you can use Ampere's Law: $$\oint \vec B\cdot \mathrm{d}\vec\ell=\mu_0I_\text{enc}.$$ Specifically, it is the form without the displacement current, and it works because you are in magnetostatics. And Rob Jeffries' answer is totally satisfactory. But to specifically address your concern with the charge build-up lets look at an example of a proper time dependent generalization of Biot-Savart.
An example of a solution to Maxwell can be provided if both the electric and magnetic field are computed as the electric and magnetic parts of the electromagnetic field given by Jefimenko's equations:
$$\vec E(\vec r,t)=\frac{1}{4\pi\epsilon_0}\int\left[\frac{\rho(\vec r',t_r)}{|\vec r -\vec r'|}+\frac{\partial \rho(\vec r',t_r)}{c\partial t}\right]\frac{\vec r -\vec r'}{|\vec r -\vec r'|^2}\; \mathrm{d}^3\vec{r}'
-\frac{1}{4\pi\epsilon_0c^2}\int\frac{1}{|\vec r-\vec r'|}\frac{\partial \vec J(\vec r',t_r)}{\partial t}\mathbb{d}^3\vec r'$$ and
$$\vec B(\vec r,t)=\frac{\mu_0}{4\pi}\int\left[\frac{\vec J(\vec r',t_r)}{|\vec r -\vec r'|^3}+\frac{1}{|\vec r -\vec r'|^2}\frac{\partial \vec J(\vec r',t_r)}{c\partial t}\right]\times(\vec r -\vec r')\mathbb{d}^3\vec r'$$
where $t_r$ is actually a function of $\vec r'$, specifically $t_r=t-\frac{|\vec r-\vec r'|}{c}.$
These reduce to Coulomb and Biot-Savart only when those time derivatives are exactly zero, which is statics. So Jefimenko is an example of proper time dependent laws for the electromagnetic field. Note that both the electric and the magnetic part of the electromagnetic field have parts that depend on the time variation of current.
When the time variation of the current is zero, the magnetic field is solely determined by the current. Full stop. Do not worry about displacement current. Instead you have to worry that for a point $\vec r$ at a time $t$ you have to consider those places $\vec r'$ that could have current and look at what the current was back at time $t_r=t-\frac{|\vec r-\vec r'|}{c}.$ And if that current was changing back then, you need to look at its time derivative back then too. But for magnetic fields that is it.
For electric fields you will care about the charge, and the rate charge is piling up, and the time rate of change of the current. If you want to use Ampere in a situation where the current never changes, then you can.
If the current changes then this will produce an additional magnetic field, and it will also produce an additional electric field. And you can compute the additional magnetic field by looking at the rate of change of the displacement current.
But Jefimenko's equations make a casual version obvious. The current and its change, both in the past, cause the magnetic field here in the present. And the fields here in the present bear some relationships to each other because of their common cause.
So if you use these equations, the change in current directly causes both electric and magnetic fields. But when the current changes at place-time $(\vec r_1,t_1)$, there is an electric and a magnetic field. But the field exists only at place-times $(\vec r_2,t_2)$ where $t_2=t_1+\frac{|\vec r_2-\vec r_1|}{c}$.