I'll use this approach:
Start with a metric that is non-singular for all $r\neq 0$.
Transform the time coordinate to get the more familiar form of an extremal black hole.
The coordinate singularity on the horizon enters in step 2, because the coordinate transform itself is singular. The fact that we started with a non-singular metric shows that the singularity on the horizon is an artifact of the coordinate system.
Non-rotating black hole with extremal charge
Let $d\Omega^2$ denote the standard metric on the unit sphere, and use the letters $w,r$ for the other two coordinates. Start with the metric
$$
dw^2-dr^2-V(r)(dw+dr)^2-r^2 d\Omega^2
\tag{1}
$$
where $V(r)$ is smooth and finite for all $r>0$. Define a function $f(r)$ by
$$
\frac{d}{dr}f(r)=\frac{V(r)}{V(r)-1},
\tag{2}
$$
and define a new coordinate $t$ by
$$
w = t + f(r).
\tag{3}
$$
Substitute (3) into (1) and use (2) to get this identity, after a little algebra:
$$
dw^2-dr^2-V(r)(dw+dr)^2-r^2 d\Omega^2
=
\big(1-V(r)\big)dt^2-\frac{dr^2}{1-V(r)}-r^2d\Omega^2.
\tag{4}
$$
The metric (1) was nonsingular for all $r>0$, but the coordinate transform (3) introduced a singularity at the value of $r$ for which $V(r)=1$, which is thus obviously only a coordinate singularity.
To apply this to the case of an extremal charged non-rotating black hole, define the function $V(r)$ by
$$
V(r) \equiv 1-\left(1-\frac{Q}{r}\right)^2.
\tag{5}
$$
Then (4) is the familiar form of the metric for the extreme black hole, and the metric (1) is clearly nonsingular for all $r>0$. Mission accomplished.
Actually, we need to be a little more careful before we conclude that (1) is well-behaved when $V(r)=1$, because the $dw^2$ term in (1) cancels when $V(r)=1$. One way to see that the metric is still nondegenerate there is to use the identity $dr^2+dr\,dw = (du^2-dw^2)/4$ with $u\equiv w+2r$.
The metric (1) is an example of a Kerr-Schild metric. This whole analysis also works for non-extremal charged black holes, just by generalizing the function (5).
Uncharged black hole with extremal rotation
The extremal rotating black hole can be handled in a similar way. For a Kerr black hole (extremal or not), the Kerr-Schild form of the metric is
$$
\newcommand{\bfu}{\mathbf{u}}
\newcommand{\bfx}{\mathbf{x}}
dw^2-d\bfx^2 - V(\bfx)\big(dw+\bfu(\bfx)\cdot d\bfx\big)^2
\tag{6}
$$
where the independent coordinates are $w$ and $\bfx=(x,y,z)$, and where the functions $\bfu=(u_x,u_y,u_z)$ and $V$ are defined by
$$
u_x+iu_y = \frac{x+iy}{r(\bfx)+ia}
\hspace{2cm}
u_z = \frac{z}{r(\bfx)}
\hspace{2cm}
V = M\nabla\cdot\bfu,
\tag{7}
$$
where $\nabla$ is the gradient with respect to $\bfx$ and where the function $r(\bfx)$ is defined implicitly by the conditions
$$
\bfu^2=1
\hspace{2cm}
r\geq 0.
\tag{8}
$$
Everything in equations (6)-(8) is nonsingular for all $r>0$, even in the extremal case $a=M$. To relate this to the Boyer-Lindquist form of the metric, define new coordinates $t,\hat x,\hat y$ by
$$
t = w - f(r)
\hspace{2cm}
\hat x = x+ay/r
\hspace{2cm}
\hat y = y-ax/r
\tag{9}
$$
with
$$
\frac{d}{dr}f(r) = \frac{2Mr}{r^2-2Mr+a^2},
\tag{10}
$$
and then express $\hat x,\hat y,z$ in terms of $r$ and angles as usual. After a lot of algebra, this should reproduce the familiar Boyer-Lindquist form of the metric. The coordinate transform (9)-(10) is singular where $r^2-2Mr+a^2=0$, which is why the resulting Boyer-Lindquist form of the metric has a coordinate singularity there, even though the original metric (6) has no such singularity.