There are several different concepts of "basis of a vector space" used in physics and rarely distinguished, in particular the context of quantum mechanics. The most important one is "Orthonormal Basis of a Hilbert space", which is different from an algebraic basis of a Hilbert space, that may or may not be orthonormal (in the finite dimensional case these concepts agree). The difference is that for an algebraic basis every vector must be a unique FINITE linear combination of basis vectors, while for a "Hilbert Space Orthonormal Basis" infinite series are allowed.

Different from these, and not bases of Hilbert space in the mathematical sense at all, are things like Dirac Deltas, plane waves and coherent states. Dirac Deltas and plane waves can be considered "generalized wavefunctions" in a formalism called "Rigged Hilbert Space" or the "Theory of Gelfand Triples" and hence also form some kind of "generalized basis". So do coherent states, which is what your question is about.

Consinder the Hilbert space $L^2(\mathbb{R})$ of square integrable complex valued functions on the real line. One has the self adjoint position and momentum operators $(X\psi)(x)=x\psi(x)$ and $(P\psi)(x)=-i\psi'(x)$ where I will ignore units entirelly. Think of $X$ and $P$ as being divided by some combination of parameters and constants of the system ($\hbar, \omega, m$...) such that they are dimensionless. Now one forms the creation and annihilation operators, familiar from the discussion of the Quantum Harmonic Oszillator: $$a^{\dagger}=Q-iP$$ $$a=Q+iP$$ Define the normalized gaussian wave function $\psi_0(x):=\frac{1}{\sqrt{2\pi}}e^{-x^2/2}$. Then define for $n\in\mathbb{N}_0$ the states $$| n\rangle:=\frac{1}{\sqrt{n!}}(a^{\dagger})^n\psi_0.$$ Note that $\langle x|0\rangle=\psi_0(x)$. These states form an orthonormal basis (up to constants, the eigenbasis of the harmonic oscillator). Now the Coherent States of the Harmonic Oscillator are all Eigenvectors of the annihilation operator $a$. Since $a$ is not hermitian, it can have complex eigenvalues.

There is a unique normalized eigenvector of $a$ for each complex number $\alpha$.

Proof. Suppose $\psi_{\alpha}\neq 0$ and $a\psi_{\alpha}=\alpha\psi_{\alpha}$. Then writing $\psi_{\alpha}=\sum_{n=0}^{\infty}c_n|n\rangle$ and applying $a$, where $a|n\rangle=\sqrt{n}|n-1\rangle$ for $n\in\mathbb{N}_{\geq1}$ and $a|0\rangle=0$ , we get $$\sum_{n=1}^{\infty}c_{n+1}\sqrt{n+1}|n\rangle=\alpha\sum_{n=0}^{\infty}c_n|n\rangle.$$ By linear indepencence, this implies $\sqrt{n+1}c_{n+1}=z c_n$, from which we obtain by induction $$c_{n}=\frac{\alpha^n} {\sqrt{n!}}c_0.$$ Hence all $c_n$ are fixed by a choice of $c_0$ and $c_0$ is fixed by requiring that $\psi_{\alpha}$ is normalized, to be $c_0:=e^{-\frac{1}{2}|\alpha|^2}$. All in all, $$\psi_{\alpha}=e^{-\frac{1}{2}|\alpha|^2}\sum_{n=0}^{\infty}\frac{\alpha^n}{\sqrt{n!}}|n\rangle.$$ Which hereby is non-zero, exists and is unique for each $\alpha$. $\blacksquare$

Why these states are called coherent states is a very interesting question, which I do not want to go into right now; let me just remark that these states are basically all displaced gaussians in position space (and hence also in momentum space) saturating the Heisenberg uncertainty relation, and their dynamical behaviour under the harmonic oscillator hamiltonian corresponds in the nicest way allowed by quantum mechanics to the classical harmonic motion. In the literature one often writes $|\alpha\rangle$ instead of $\psi_{\alpha}$ but I find that somewhat confusing (what is $|5\rangle$? Is it $|n=5\rangle$ or $|\alpha=5\rangle$?)

Now to answer your question: The coherent states form a "generalized sort of overcomplete set" in the sense that

Theorem. For every wave function $\phi$ we have $$\phi=\frac{1}{\pi}\int_{-\infty}^{\infty}dx\int_{-\infty}^{\infty}dy \langle \psi_{(x+iy)}|\phi\rangle\psi_{(x+iy)}$$

which looks similar to an expansion in a basis, exept that instead of a sum we have an integral over the entire complex plane. Note that even a coherent state can be expanded like this, where the "coefficients" (scalar products between coherent states) will be non-zero; no deltas involved here. Coherent states are not orthogonal, although if $|\alpha- \beta|$ is large, $|\langle \psi_{\alpha}|\psi_{\beta}\rangle|$ is small. This is why people say that coherent states are not linearly independant, since one can be expanded in terms of others. Needless to say, any finite set of coherent states is still linearly independent.

To get a notion of uniqueness one has to introduce spaces of "square integrable" holomorphic functions, called "Bargmann-Segal spaces", but I will not go into that here. Applications of this are mostly in quantum optics but there are also some uses in areas like quantum gravity.

References:

Bongaarts, P. (2015). Quantum Theory A Mathematical Approach. Springer International Publishing, Cham.

Hall, B. C. (2000). Holomorphic methods in mathematical physics. Contemporary Mathematics, 260:1-59. arXiv:quant-ph/9912054. (Good introduction)

Bargmann, V. (1962). Remarks on a hilbert space of analytic functions. Proceedings of the National Academy of Sciences of the United States of America, 48(2):199-204.

Zhu, K. (2012). Graduate Texts in Mathematics: Analysis on Fock Spaces. Springer US. (Some mathematiciens call "Bargmann-Segal spaces" Fock spaces, while for physicists, Fock spaces are tensor algebras over a Hilbert space. There are relations between these, see Stochel, J. (2005). A note on inductive limit model of bargmann space of infnite order. Opuscula Math., 25(1):139-148.)