There are at least three notions of basis depending on the mathematical structure you are considering. I will quickly discuss three cases relevant in physics (topological vector spaces are relevant too, but I will not consider them for the shake of brevity). 

(1) **Pure algebraic structure** (i.e. vector space structure over the field $\mathbb K=$ $\mathbb R$ or $\mathbb C$, actually the definition applies also to modules). 

**Basis in the sense of Hamel.**

*Given a vector space $V$ over the field $\mathbb K$, a set $B \subset V$ is called **algebraic basis** or **Hamel basis**, if its elements are linearly independent and every $v \in V$ can be decomposed as: $$v = \sum_{b \in B} c_b b$$* 
*for a **finite** set of **non-vanishing** numbers $c_b$ in $\mathbb K$ depending on $v$.*


**Completeness** of $B$ means here that the set of **finite linear** combinations of elements in $B$ includes (in fact coincide to) the whole space $V$. 

**Remarks.** 

  - This definition applies to infinite dimensional vector spaces, too. Existence of algebraic bases arises from Zorn's lemma. 

  - It is possible to prove that all algebraic bases have same cardinality. 

  - Decomposition of $v$ over the basis $B$ turns out to be **unique**.

(2) **Banach space structure** (i.e. the vector space over $\mathbb K$ admits a norm $||\:\:|| : V \to \mathbb R$ and it is complete with respect to the metric topology induced by that norm). 

**Basis in the sense of Schauder.**


*Given an infinite dimensional Banach space $V$ over the field $\mathbb K = \mathbb C$ or $\mathbb R$, a **countable ordered**  set $B := \{b_n\}_{n\in \mathbb N} \subset V$ is called **Schauder basis**, if every $v \in V$ can be **uniquely** decomposed as: $$v = \sum_{n \in \mathbb N} c_n b_n\quad (2)$$* 
for a set, generally infinite,  of numbers $c_n \in \mathbb K$ depending on $v$ where the convergence of the sum is referred both to the Banach space topology and to the order used in labelling $B$. Identity (2) means:
$$\lim_{N \to +\infty} \left|\left|v - \sum_{n=1}^N c_{n} b_n\right|\right| =0$$
 
**Completeness** of $B$ means here that the set of **countably infinite** linear combinations of elements in $B$ includes (in fact coincide to) the whole space $V$. 

**Remarks.** 

 - The elements of a Schauder basis are linearly independent (both for finite and infinite linear combinations).

 - An infinite dimensional  Banach space also admits Hamel bases since it is a vector space too. However it is possible to prove that Hamel bases are always uncountable differently form Schauder ones.

 - Not all infinite dimensional Banach space admit Schauder bases. A necessary, but not sufficient, condition is that the space must be separable (namely it contains a dense countable subset).

(3) **Hilbert space structure** (i.e. the vector space over $\mathbb K$ admits a scalar product $\langle \:\:| \:\:\rangle : V \to \mathbb K$ and it is complete with respect to the metric topology induced by the norm
$||\:\:||:= \sqrt{\langle \:\:| \:\:\rangle }$). 

**Basis in the sense of Hilbert (Riesz- von Neumann).**


*Given an infinite dimensional Hilbert space $V$ over the field $\mathbb K = \mathbb C$ or $\mathbb R$, a  set $B \subset V$ is called **Hilbert basis**, if the following conditions are true:*

*(1) $\langle z | z \rangle =1$ and $\langle z | z' \rangle =0$
if $z,z' \in B$ and $z\neq z'$, i.e. $B$ is an **orthonormal system**;*

*(2) if $\langle x | z \rangle =0$ for all $z\in B$ then $x=0$ (i.e. $B$ is maximal with respect to the orthogonality requirment).*

Hilbert bases are also called **complete orthonormal systems** (of vectors).

The relevant properties of Hilbert bases are fully encompassed within the following pair of propositions.

**Proposition.**  *If $H$ is a (complex or real) Hilbert space and $B\subset H$ is an orthonormal system (not necessarily complete) then, for every $x \in H$ the set of **non-vanishing**  elements $\langle x| z \rangle$ with $z\in B$ is at most countable.* 


**Theorem.** *If $H$ is a (complex or real) Hilbert space and $B\subset H$ is a Hilbert basis, then the following identities hold, where the order employed in computing the infinite sums (in fact countable sums due to the previous proposition) does not matter:*
$$||x||^2 = \sum_{z\in B} |\langle x| z\rangle|^2\:, \qquad \forall x \in H\:,\qquad (3)$$

$$\langle x| y \rangle = \sum_{z\in B} \langle x|z \rangle \langle z| y\rangle\:, \qquad \forall x,y \in H\:,\qquad (4)$$


$$\lim_{n \to +\infty} \left|\left| x - \sum_{n=0}^N z_n \langle z_n|x \rangle  \right|\right| =0\:, \qquad \forall x \in H \:,\qquad (5)$$

*where the $z_n$ are the elements in $B$ with $\langle z|x\rangle \neq 0$.*

*If an orthonormal system verifies one of the three identities above then it is a Hibertian basis.*


**Completeness** of $B$ means here that the set of **infinite** linear combinations of elements in $B$ includes (in fact coincide to) the whole space $H$. 

**Remarks.** 

 - The elements of a Hilbert basis are linearly independent (both for finite and infinite linear combinations).

 - All Hilbert spaces admit corresponding Hilbert bases. In a fixed Hilbert space all Hilbert bases have the same cardinality. 

 - An infinite dimensional  Hilbert space is separable (i.e. it contains a dense countable subset) if and only if it admits a countable Hilbert basis. 

 - An infinite dimensional  Hilbert space also admits Hamel bases, since it is a vector space as well. 

 - In a separable infinite dimensional Hilbert space a Hilbert basis is also a Schauder basis (the converse is false).

 
**FINAL COMMENTS.** 

 - Identities like this:
$$ \sum_n \phi_n(x) \phi_n^*(x') = \frac{1}{w(x)}\delta(x-x') \,\qquad (6)$$
stay for the **completeness property** of a Hilbert basis in $L^2(X, w(x)dx)$: Identity (6) is nothing but a formal version of equation (4) above.
However such an identity is completely formal and, in general it does not hold if $\{\phi_n\}$ is a Hilbert basis of $L^2(X, w(x)dx)$ (also because the value of $\phi_n$ at $x$ does not make any sense in $L^2$ spaces, as its elements are defined up to zero measure sets and $\{x\}$ has zero measure).  That identity sometime holds rigorously if (1) the functions $\phi_n$ are sufficiently regular and (2) the identity is understood in **distributional** sense, working with suitably smooth test functions like  ${\cal S}(\mathbb R)$ in $\mathbb R$.   

 - In $L^2(\mathbb R, d^nx)$ spaces all Hilbert bases are **countable**. Think of the basis of eigenvectors of the Hamiltonian operator of an Harmonic oscillator in $L^2(\mathbb R)$ (in $\mathbb R^n$ one may use a $n$ dimensional harmonic oscillator). However, essentially for for practical computations it is convenient  also speaking of *formal eigenvectors* of, for example,  the position operator: $|x\rangle$. In this case, $x \in \mathbb R$ so it could seem that $L^2(\mathbb R)$ admits also uncountable bases. It is false! $\{|x\rangle\}_{x\in \mathbb R}$ is **not** an orthonormal basis. It is just a *formal object*, (very) useful in computations. 
If you want to make rigorous these objects, you should picture the space of the states as a **direct integral** over $\mathbb R$ of finite dimensional spaces $\mathbb C$,  or as a **rigged Hilbert space**. In both cases however $\{|x\rangle\}_{x\in \mathbb R}$ is not an orthonormal Hilbertian basis. And $|x\rangle$ does not belong to $L^2(\mathbb R)$.

 - Hilbert bases are not enough to state and prove the **spectral decomposition theorem** for **normal** operators in a complex Hilbert space. Normal operators $A$ are those verifying $AA^\dagger= A^\dagger A$, unitary and self-adjoint ones are particular cases.
The notion of Hilbert basis  is however enough for stating the said theorem for normal **compact** operators or normal operators whose resolvent is compact. In that case, the spectrum is a **pure point spectrum** (with only a possible point in the  continuous part of the spectrum). It happens, for example, for the Hamiltonian operator of the harmonic oscillator. In general one has to introduce the notion of **spectral measure** or PVM (projector valued measure) to treat the general case.