In undergrad physics, when analyzing an LR circuit, it is often considered that Kirchoff rule holds. However, as far as I understand, Kirchoff rule only holds when E field is conservative (curl of E is 0, there is no change of flux). In a coil L, there is obviously a self-induced emf which makes closed loop integral E.dl non zero. My question is, does that mean we can only apply Faraday's law but not Kirchoff rule here.
No, Kirchhoff's Voltage Law is applicable even for inductors. This is because the effect of non-conservative field is limited to the inductor and is accounted for accurately by the potential drop $LdI/dt$ on the inductor; validity of standard formulae for potential drops on other elements is not affected.
Problem would arise if there was an external induced electric field acting on some parts of the circuit (or everywhere in it), say due to moving magnet near the circuit, or due to electric field of another circuit with high mutual inductance (like in transformer).
Then, KVL would be still valid, but would be hard to use, because the additional emf would invalidate the familiar formulae for potential drop on standard elements like $0$ for wire, $RI$ for resistor, $LdI/dt$ for inductor.
However, the original Kirchhoff's second circuital law would still be applicable; it is formulated using circuit emfs and $RI$'s: sum of all emf's acting in a closed loop equals sum of terms $RI$ for all loop elements.
So the additional external emf could be accounted for by an additional EMF term in the equation.
Another question, even when there is no coil (only R's for example), in practice as we turn on the power supply, the current will create a B field inside the wire loop which in turn creates a change in magnetic flux. As a result, we again have a non-conservative E field here. Is it correct to say Kirchoff rule might never be applicable in practice? (at least to an absolute accuracy)
No. KVL in terms of potential drops is always true, but sometimes it is not applicable because we can't express those potential drops. Then we can sometimes turn back to the original formulation, which is applicable in this case.