There are at least three notions of basis depending on the mathematical structure you are considering. I will quickly discuss three cases relevant in physics (topological vector spaces are relevant too, but I will not consider them for the shake of brevity). 

(1) **Pure algebraic structure** (i.e. vector space structure over the field $\mathbb K=$ $\mathbb R$ or $\mathbb C$, actually the definition applies also to modules). 

**Basis in the sense of Hamel.**

*Given a vector space $V$ over the field $\mathbb K$, a set $B \subset V$ is called **algebraic basis** or **Hamel basis**, if its elements are linearly independent and every $v \in V$ can be decomposed as: $$v = \sum_{b \in B} c_b b$$* 
*for a **finite** set of **non-vanishing** numbers $c_b$ in $\mathbb K$ depending on $v$.*


**Completeness** of $B$ means here that the set of **finite linear** combinations of elements in $B$ includes (in fact coincide to) the whole space $V$. 

**Remarks.** 

  - This definition applies to infinite dimensional vector spaces, too. Existence of algebraic bases arises fromm Zorn's lemma. 

  - It is possible to prove that all algebraic bases have same cardinality. 

  - Decomposition of $v$ over the basis $B$ turns out to be **unique**.

(2) **Banach space structure** (i.e. the vector space over $\mathbb K$ admits a norm $||\:\:|| : V \to \mathbb R$ and it is complete with respect to the metric topology induced by that norm). 

**Basis in the sense of Schauder.**


*Given an infinite dimensional Banach space $V$ over the field $\mathbb K = \mathbb C$ or $\mathbb R$, a **countable ordered**  set $B := \{b_n\}_{n\in \mathbb N} \subset V$ called **Schauder basis**, if every $v \in V$ can be **uniquely** decomposed as: $$v = \sum_{n \in \mathbb N} c_n b_n\quad (2)$$* 
for a set, generally infinite,  of numbers $c_n \in \mathbb K$ depending on $v$ where the convergence of the sum is referred both to the Banach space topology and to the order used in labelling $B$. Identity (2) means:
$$\lim_{N \to +\infty} \left|\left|v - \sum_{n=1}^N c_{n} b_n\right|\right| =0$$
 
**Completeness** of $B$ means here that the set of **countably infinite** linear combinations of elements in $B$ includes (in fact coincide to) the whole space $V$. 

**Remarks.** 

 - The elements of a Schauder basis are linearly independent (both for finite and infinite linear combinations).

 - An infinite dimensional  Banach space also admits Hamel bases since it is a vector space too. However it is possible to prove that Hamel bases are always uncountable differently form Schauder ones.

 - Not all infinite dimensional Banach space admit Schauder bases. A necessary, but not sufficient, condition is that the space must be separable (namely it contains a dense countable subset).

(3) **Hilbert space structure** (i.e. the vector space over $\mathbb K$ admits a scalr product $\langle \:\:| \:\:\rangle : V \to \mathbb K$ and it is complete with respect to the metric topology induced by the norm
$||\:\:||:= \sqrt{\langle \:\:| \:\:\rangle }$). 

**Basis in the sense of Hilbert (Riesz- von Neumann).**


*Given an infinite dimensional Hilbert space $V$ over the field $\mathbb K = \mathbb C$ or $\mathbb R$, a  set $B \subset V$ called **Hilbert basis**, if the following conditions are true:*

*(1) $\langle z | z \rangle =1$ and $\langle z | z' \rangle =0$
for $z,z' \in B$ i.e. $B$ is an **orthonormal system**;*

*(2) if $\langle x | z \rangle =0$ for all $z\in B$ then $x=0$ (i.e. $B$ is maximal with respect to the orthogonality requirment).*

Hilbert bases are also called **complete orthonormal systems** (of vectors).

The relevant properties of Hilbert bases are fully encompassed within the following pair of propositions.

**Proposition.**  *If $H$ is a (complex or real) Hilbert space* and $B\subset H$ is an orthonormal system (not necessarily complete) then, for every $x \in H$ the set of **non-vanishing**  elements $\langle x| z \rangle$ with $z\in B$ is at most countable.* 


**Theorem.** *If $H$ is a (complex or real) Hilbert space* and $B\subset H$ is a Hilbert basis then the following identities hold where the order used in computing the infinite sums (in fact countable sums for the previous proposition) does not matter:*
$$||x||^2 = \sum_{z\in B} |\langle x| z\rangle|^2\:, \qquad \forall x \in H$$

$$\langle x| y \rangle = \sum_{z\in B} \langle x|z \rangle \langle z| y\rangle\:, \qquad \forall x,y \in H\:,$$


$$\lim_{n \to +\infty} \left|\left| x - \sum_{n=0}^N z_n \langle z_n|x \rangle  \right|\right| =0\:, \qquad \forall x \in H \:,$$

*where the $z_n$ are the elements in $B$ with $\langle z|x\rangle \neq 0$.*

*If an orthonormal system verifies one of the three identities above then it is a Hibertian basis.*


**Completeness** of $B$ means here that the set of **infinite** linear combinations of elements in $B$ includes (in fact coincide to) the whole space $H$. 

**Remarks.** 

 - The elements of a Hilbert basis are linearly independent (both for finite and infinite linear combinations).

 - All Hilbert space admit Hilbert bases. In a Hilbert space all Hilbert bases have the same cardinality. 

 - An infinite dimensional  Hilbert space is separable if and only if admits a countable Hilbert basis. 

 - An infinite dimensional  Hilbert space also admits Hamel bases since it is a vector space. However it is possible to prove that Hamel bases are uncountable differently form Schauder ones.

 - In a separable infinite dimensional Hilbert space a Hilbert basis is also a Schauder basis (the converse is false).

 
**FINAL COMMENT.** Identities like this:
 
$$ \sum_n \phi_n(x) \phi_n^*(x') = \frac{1}{w(x)}\delta(x-x') \,.$$

stay for the completeness property of a Hilbert basis in $L^2(X, w(x)dx)$. However such an identity is completely formal and, in general it does not hold if $\{\phi_n\}$ is ah Hilbert basis of $L^2(X, w(x)dx)$ (also because the value of $\phi_n$ at $x$ does not make any sense in $L^2$ spaces, as its elements are defined up to zero measurwe sets and $\{x\}$ has zero measure.  That identity sometime holds rigorously if (1) the functions $\phi_n$ are sufficiently regular and (2) the identity is understood in **distributional** sense, working with suitably smooth test functions like  ${\cal S}(\mathbb R)$ in $\mathbb R$.