Is tangential component of $\mathbf{B}$ undefined at the boundary of two media? Tangential component of $\mathbf{B}$ is discontinuous at the boundary of two media. Does this mean that tangential component of $\mathbf{B}$ is undefined at the boundary of two media?
If yes, then:
$\mathbf{B}$ is undefined at the boundary of two media.
$\nabla \cdot\mathbf{B}$ is undefined at the boundary of two media.
This contradicts $\nabla \cdot\mathbf{B}=0$ everywhere. How to get out of this difficulty?
If no, then:
What is the value of tangential component of $\mathbf{B}$ at the boundary of two media?
 A: No the tangential component of B is well defined. It is just discontinuous. As it is in the direction perpendicular to the tangential component, the discontinuity does not affect $\vec \nabla \cdot \vec B$, as it happens in the direction perpendicular to $\vec B$. 
Note that in this case many quantities are discontinuous: current, atomic density, electron density, dielectric response, to name a few. Of course the discontinuity is a limit of your model of the media. At atomic scale there is no discontinuity. 
A: 
Does this mean that tangential component of  is undefined at the boundary of two media?

An infinitely sharp boundary with a delta function surface current?  Yes.

This contradicts ∇⋅=0 everywhere. How to get out of this difficulty?

What's the difficulty?  Maxwell's equations are differential equations for vector fields, and hold on a particular domain.  In the model you're describing, the sharp boundary isn't actually in the domain of $\mathbf B$.

If you're uncomfortable with this, there are workarounds.  If you define the tangential component of $\mathbf B$ on the boundary to be the average of the tangential components on either side, and interpret the derivatives in Maxwell's equations in a weak sense, then everything is well-defined.
Alternatively, because nothing in nature is ever infinitely sharp, you could replace the sharp boundaries and surface currents with things like bump functions, which would eliminate discontinuities by smoothing everything out.
A: The interface condition for the tangential component of the magnetic field is derived from the Ampere-Maxwell equation. Here is a detailed derivation on the same.
                                                                        
A discontinuity in the tangential component of the magnetic field intensity at the boundary must be supported by surface current flowing in a direction perpendicular to that component of the field. For an infinitesimally thin interface between two macroscopic regions (labelled 1 and 2) carrying as surface charge density $K$, denoting the unit normal from region 1 and 2 as $\hat{n}_1$ and $\hat{n}_2$ respectively, it is usually stated as $$
\hat{n}_2\times [\vec{B_1} - \vec{B_2}] = \mu \vec{K}$$
Suppose there is no surface current. Then the tangential component of the magnetic field intensity is continuous across the boundary unless the permeabilities are unequal. Also, note that the particular expression for discontinuity works for only to an interface which is at rest in the frame of reference where the fields are measured.
A: The tangential boundary condition for ${\bf B}$ can be obtained from the tangential boundary condition for ${\bf H}$ which is a consequence of Ampère's law, by taking a small rectangular circuit with two sides parallel to the surface:
$$
{\bf n \times( H_2 - H_1 )= J_s}~~~~~~~~~~~~~[1]
$$
where $J_s$ is the surface current density which may be present on the surface separating  region 1 and region 2. The unit vector {\bf n} is normal to the sepaation surface and points from region 1 to region 2.
This relation for ${\bf H}$ can be rewritten in term of ${\bf B}$ by using the relation between  ${\bf H}$ and  ${\bf B}$ fields in the two regions. For example, for linear media where ${\bf B}= \mu {\bf H}$, eq.[1] becomes
$$
{\bf n} \times( { \bf B_2}/\mu_2 - {\bf B_1}/\mu_1 {\bf )= J_s}~~~~~~~~~~~~~[2]
$$
As you can see, the (possible) jump of ${\bf B}$ is determined by this condition.
A: The tangential component of the B-field is undefined at an infinitely sharp interface, but the divergence of the B-field is still zero.
Consider a simple example. An interface between two media with different permeabilities. The interface is defined by the plane $z=0$.  The B-field either side of the interface is wholly parallel to the plane of the interface. Let us also say there are no moving charges anywhere.
Maxwell's equations (Ampere's law) can be used to show that $\vec{H}_1= \vec{H}_2$, where in both cases, the H-fields have components along the $x$ and $y$ axes.
If we are dealing with linear, isotropic, homogeneous media then $\vec{B} =\mu \vec{H}$ and so $\mu_2 \vec{B}_1 = \mu_1 \vec{B}_2$ and the B-field changes, as a step function, as we move from one medium to another.
This step function is an artefact of the model for the interface. If the interface is modelled as a step function we should not be surprised to find step functions in other quantities.
However that does not mean that $\nabla \cdot \vec{B}$ is undefined,  since it must satisfy the "solenoidal law" of Maxwell's equations.
The B-field only changes discontinuously along the $z$-axis, and since $B_z = 0$, then $\partial B_z/\partial z =0$ and the divergence of the B-field is still zero everywhere, as it must be.
The introduction of a z-component to the field either side of the interface will make no difference, because we say that this component is continuous by demanding that the divergence of the B-field is zero at the interface!
Ditto for adding surface currents. They only change the B-fields along the x and y axes and in a divergence-free manner.
