There are at least three notions of basis depending on the mathematical structure you are considering. I will quickly discuss three cases relevant in physics (topological vector spaces are relevant too, but I will not consider them for the shake of brevity).
(1) Pure algebraic structure (i.e. vector space structure over the field $\mathbb K=$ $\mathbb R$ or $\mathbb C$, actually the definition applies also to modules).
Basis in the sense of Hamel.
Given a vector space $V$ over the field $\mathbb K$, a set $B \subset V$ is called algebraic basis or Hamel basis, if its elements are linearly independent and every $v \in V$ can be decomposed as: $$v = \sum_{b \in B} c_b b$$ for a finite set of non-vanishing numbers $c_b$ in $\mathbb K$ depending on $v$.
Completeness of $B$ means here that the set of finite linear combinations of elements in $B$ includes (in fact coincide to) the whole space $V$.
Remarks.
This definition applies to infinite dimensional vector spaces, too. Existence of algebraic bases arises fromm Zorn's lemma.
It is possible to prove that all algebraic bases have same cardinality.
Decomposition of $v$ over the basis $B$ turns out to be unique.
(2) Banach space structure (i.e. the vector space over $\mathbb K$ admits a norm $||\:\:|| : V \to \mathbb R$ and it is complete with respect to the metric topology induced by that norm).
Basis in the sense of Schauder.
Given an infinite dimensional Banach space $V$ over the field $\mathbb K = \mathbb C$ or $\mathbb R$, a countable ordered set $B := \{b_n\}_{n\in \mathbb N} \subset V$ called Schauder basis, if every $v \in V$ can be uniquely decomposed as: $$v = \sum_{n \in \mathbb N} c_n b_n\quad (2)$$ for a set, generally infinite, of numbers $c_n \in \mathbb K$ depending on $v$ where the convergence of the sum is referred both to the Banach space topology and to the order used in labelling $B$. Identity (2) means: $$\lim_{N \to +\infty} \left|\left|v - \sum_{n=1}^N c_{n} b_n\right|\right| =0$$
Completeness of $B$ means here that the set of countably infinite linear combinations of elements in $B$ includes (in fact coincide to) the whole space $V$.
Remarks.
The elements of a Schauder basis are linearly independent (both for finite and infinite linear combinations).
An infinite dimensional Banach space also admits Hamel bases since it is a vector space too. However it is possible to prove that Hamel bases are always uncountable differently form Schauder ones.
Not all infinite dimensional Banach space admit Schauder bases. A necessary, but not sufficient, condition is that the space must be separable (namely it contains a dense countable subset).
(3) Hilbert space structure (i.e. the vector space over $\mathbb K$ admits a scalr product $\langle \:\:| \:\:\rangle : V \to \mathbb K$ and it is complete with respect to the metric topology induced by the norm $||\:\:||:= \sqrt{\langle \:\:| \:\:\rangle }$).
Basis in the sense of Hilbert (Riesz- von Neumann).
Given an infinite dimensional Hilbert space $V$ over the field $\mathbb K = \mathbb C$ or $\mathbb R$, a set $B \subset V$ called Hilbert basis, if the following conditions are true:
(1) $\langle z | z \rangle =1$ and $\langle z | z' \rangle =0$ for $z,z' \in B$ i.e. $B$ is an orthonormal system;
(2) if $\langle x | z \rangle =0$ for all $z\in B$ then $x=0$ (i.e. $B$ is maximal with respect to the orthogonality requirment).
Hilbert bases are also called complete orthonormal systems (of vectors).
The relevant properties of Hilbert bases are fully encompassed within the following pair of propositions.
Proposition. If $H$ is a (complex or real) Hilbert space and $B\subset H$ is an orthonormal system (not necessarily complete) then, for every $x \in H$ the set of non-vanishing elements $\langle x| z \rangle$ with $z\in B$ is at most countable.*
Theorem. If $H$ is a (complex or real) Hilbert space and $B\subset H$ is a Hilbert basis then the following identities hold where the order used in computing the infinite sums (in fact countable sums for the previous proposition) does not matter:* $$||x||^2 = \sum_{z\in B} |\langle x| z\rangle|^2\:, \qquad \forall x \in H$$
$$\langle x| y \rangle = \sum_{z\in B} \langle x|z \rangle \langle z| y\rangle\:, \qquad \forall x,y \in H\:,$$
$$\lim_{n \to +\infty} \left|\left| x - \sum_{n=0}^N z_n \langle z_n|x \rangle \right|\right| =0\:, \qquad \forall x \in H \:,$$
where the $z_n$ are the elements in $B$ with $\langle z|x\rangle \neq 0$.
If an orthonormal system verifies one of the three identities above then it is a Hibertian basis.
Completeness of $B$ means here that the set of infinite linear combinations of elements in $B$ includes (in fact coincide to) the whole space $H$.
Remarks.
The elements of a Hilbert basis are linearly independent (both for finite and infinite linear combinations).
All Hilbert space admit Hilbert bases. In a Hilbert space all Hilbert bases have the same cardinality.
An infinite dimensional Hilbert space is separable if and only if admits a countable Hilbert basis.
An infinite dimensional Hilbert space also admits Hamel bases since it is a vector space. However it is possible to prove that Hamel bases are uncountable differently form Schauder ones.
In a separable infinite dimensional Hilbert space a Hilbert basis is also a Schauder basis (the converse is false).
FINAL COMMENT. Identities like this:
$$ \sum_n \phi_n(x) \phi_n^*(x') = \frac{1}{w(x)}\delta(x-x') \,.$$
stay for the completeness property of a Hilbert basis in $L^2(X, w(x)dx)$. However such an identity is completely formal and, in general it does not hold if $\{\phi_n\}$ is ah Hilbert basis of $L^2(X, w(x)dx)$ (also because the value of $\phi_n$ at $x$ does not make any sense in $L^2$ spaces, as its elements are defined up to zero measurwe sets and $\{x\}$ has zero measure. That identity sometime holds rigorously if (1) the functions $\phi_n$ are sufficiently regular and (2) the identity is understood in distributional sense, working with suitably smooth test functions like ${\cal S}(\mathbb R)$ in $\mathbb R$.