This is how I think of it;
We are talking about a reversible process here, when the relation is an equailty:
$$ S_2-S_1=\int_1^2 \left( \frac{\delta Q}{T} \right)_\text{rev.} $$
it is valid only when the heat transfer (and the process itself, actually) is reversible. For this heat transfer to be reversible, it must be a quasi-equilibrium process. That means that if the reservoir temperature is $T_L$, the system temperature (or the component if it is a flow process) should be $T_L \pm \delta T$ at every instant, where $\delta T$ is an infinitesimal value. Hence, I consider the temperature of the system, surroundings or the boundary approximately the same;
$$T_\text{sys.} \approx T_\text{boundary} \approx T_\text{surr.} \approx T$$
The whole confusion originates from picturing something like $T_\text{sys.}=400K$ and $T_\text{surr.}=300K$ (including me). However, that is not a reversible process by definition; there is a finite temperature difference. And indeed, that is when the formula turns into an inequality, because a net positive entropy generation occurs due to the irreversibility. Depending on the heat transfer direction and the temperature value, you get the following if you do the math:
$$ S_2-S_1>\int_1^2 \left( \frac{\delta Q}{T} \right)_\text{irrev.} $$
or
$$ S_2-S_1=\int_1^2 \left( \frac{\delta Q}{T} \right)_\text{irrev.}+\Delta S_\text{gen.} $$
where $\Delta S_\text{gen.}$ is strictly greater than zero.
The formula does not explicitly tell you "just put that temperature into the integral as this function of $\delta Q$ and you will get the entropy difference" in case of an irreversible process. You can not integrate $\delta Q=TdS$ and find the heat transfer $Q$ when the process is irreversible, this is why reversible processes are drawn with a solid line and irreversible processes are drawn with dashed lines.
The formula is often given as
$ S_2-S_1\ge\int_1^2 \frac{\delta Q}{T} $
which I think is very vague without putting some thought into it.
