Perturbative expansion of the metric and its inverse As I understand it, in the context of cosmological perturbation theory, one expands the metric $g_{\mu\nu}$ around some background metric (in this case the Minkowski metric) such that $$g_{\mu\nu}=\eta_{\mu\nu}+\kappa h_{\mu\nu}$$ where $\kappa<<1$ is a dimensionless parameter, and $h_{\mu\nu}$ is a symmetric tensor - a perturbation of the background metric $\eta_{\mu\nu}$.
Given this, my question is, how does one obtain the inverse metric $g^{\mu\nu}$? I read in some notes (e.g. here, top of page 2, and here, top of page 4) that it is given by $$g^{\mu\nu}=\eta^{\mu\nu}-\kappa h^{\mu\nu}+\kappa^{2}h^{\mu}_{\;\lambda}h^{\mu\lambda}+\cdots$$ Now I know how to get the expression to first-order by writing $g^{\mu\nu}=\eta^{\mu\nu}+\delta g^{\mu\nu}$, and then using that $$\delta g^{\mu\nu}=-g^{\mu\lambda}\delta g_{\lambda\sigma}g^{\sigma\nu}=-\kappa g^{\mu\lambda}h_{\lambda\sigma}g^{\sigma\nu}=-\kappa \eta^{\mu\lambda}\eta^{\sigma\nu}h_{\lambda\sigma}+\mathcal{O}(\kappa^{2})$$ However, I'm unsure how to obtain the higher order terms. Furthermore, how can one justify raising and lowering the indices of $h_{\mu\nu}$ with $\eta_{\mu\nu}$ if one includes such higher order terms?
 A: One particularly effective and quick way to write this is to write the metric as $g=\eta+\kappa h$, so that
$$g^{-1}=(\eta+\kappa h)^{-1}=\eta^{-1}(\textbf{1}+\kappa h\eta^{-1})^{-1}$$
Then we just use the expansion
$$(\textbf{1}+\epsilon\textbf{A})^{-1}=\textbf{1}-\epsilon\textbf{A}+\epsilon^2\textbf{A}^2+\cdots,$$
which holds for matrices just as it does for numbers. The desired result is found immediately, as well as higher order terms.
A: This is a relatively old question which lacks a formally complete answer. On finding myself in the need of the inverse of a metric and on  not being able to find a proper treatment elsewhere (on casual browsing), I have decided to put a proper formal treatment here. 
Following the treatment given here, one can (super-)easily derive the inverse metric to all order of perturbation theory without using ad-hoc relations. I've arranged the following in three steps.
Step - 1: Correct Statement of Problem
The metric whose inverse we intend to determine must be written in a more formal fashion:
$$ g_{\mu\nu} = \eta_{\mu\nu} + \epsilon \  ^{(1)}h_{\mu\nu} + \frac{\epsilon^2}{2!} \  ^{(2)}h_{\mu\nu} + \cdots $$
For later convenience, we move all perturbations into $H_{\mu\nu}$:
$$g_{\mu\nu} = \eta_{\mu\nu} + H_{\mu\nu} $$
This way of stating the problem is essentially different than stated by OP in the question. I hope the notation does not need any explanations. 
Step-2: And the inverse is
Let us write the inverse as: b
$$ g^{\mu\nu} = (g_{\mu\nu})^{-1}$$
$$ = \eta^{\mu \alpha} \ (\delta{^\alpha_\nu} + \eta^{\alpha\beta}H_{\beta\nu})^{-1}$$
We first note, that we can contract the background metric inside the brackets: $ H{^\alpha_\nu} = \eta^{\alpha\beta}H_{\beta\nu}$. Further, to deal with the brackets, as suggested by Bob in another response, we use the binomial expansion:
$$ (1+x)^{-1} = 1 - x + x^2 -x^3 +\cdots  $$
And, after a few steps of index gymnastics we reach:
$$ g^{\mu\nu} = \eta^{\mu\nu} - H^{\mu\nu} + H^{\mu\rho}H{_\rho^\nu} - H^{\mu\rho}H{_\rho^\beta}H{_\beta^\nu} + \cdots $$
Are we done?
Step-3: The expansion parameter
The beauty of this arrangement lies in the following realization:
$$   H^{\mu\nu} \xrightarrow{\text{can only give rise to terms with}} \epsilon^1, \epsilon^2, \epsilon^3 \cdots$$
$$  H^{\mu\rho}H{_\rho^\nu} \xrightarrow{\text{can only give rise to terms with}} \epsilon^2, \epsilon^3, \epsilon^4 \cdots        $$
$$H^{\mu\rho}H{_\rho^\beta}H{_\beta^\nu}\xrightarrow{\text{can only give rise to terms with}}   \epsilon^3, \epsilon^4, \epsilon^5 \cdots $$
Hence to get to a useful expression of the inverse, we must arrange the inverse in powers of $\epsilon$.
Doing a bit of work, we get following terms at order $\epsilon^n$:
(note overall sign comes from the last equation in step-2)


*

*$n=0$
$$\frac{1}{0!}(\eta^{\mu \nu}$$

*$n=1$
$$ \frac{1}{1!}(- h^{1\mu \nu}) $$

*$n=2$
$$ \frac{1}{2!}(2 h^{1}{}_{a}{}^{\nu} h^{1\mu a} -  h^{2\mu \nu}) $$

*$n=3$
$$\frac{1}{3!}( -6 h^{1}{}_{a}{}^{b} h^{1}{}_{b}{}^{\nu} h^{1\mu a} + 3 h^{1\mu a} h^{2}{}_{a}{}^{\nu} + 3 h^{1}{}_{a}{}^{\nu} h^{2\mu a} -  h^{3\mu \nu})$$


As should be obvious on carefully following the above treatment, the final answer neatly looks like:
$$ g^{\mu\nu} = \eta^{\mu \nu} -  \epsilon h^{1\mu \nu} + \tfrac{1}{2} \epsilon^2 (2 h^{1}{}_{a}{}^{\nu} h^{1\mu a} -  h^{2\mu \nu}) + \tfrac{1}{6} \epsilon^3 (-6 h^{1}{}_{a}{}^{b} h^{1}{}_{b}{}^{\nu} h^{1\mu a} + 3 h^{1\mu c} h^{2}{}_{c}{}^{\nu} + 3 h^{1}{}_{d}{}^{\nu} h^{2\mu d} -  h^{3\mu \nu})$$
A: For example, in gravitational wave theory to construct the Pseudo-energy momentum tensor a la Issacson tensor you actually need a perturbed generic background to second order. So let be $g_{ab}(\lambda)$ one.parameter family in the way that $$g_{ab}(\lambda)=\tilde{g}_{ab}+\lambda h^{1}_{ab}+\frac{\lambda^2}{2!} h^2_{ab}$$  then clearly the inverse is gonna be given by $$g^{ab}\equiv(g_{ab}(\lambda))^{-1}$$ so at first order in $\lambda$ the we need to perform the first derivative respect to the parameter $$\frac{d}{d\lambda}(g_{ab}(\lambda))^{-1}\lvert_{\lambda=0}=-\tilde{g}^{ac}\tilde{g}^{bd}h^1_{cd}=-h^{ab}$$in the same way for second order in  $\lambda$ you need the second derivative
$$\frac{d^2}{d\lambda^2}(g_{ab}(\lambda))^{-1}\lvert_{\lambda=0}=-\left(\frac{d}{d\lambda}(g_{ab}(\lambda))^{-2}\frac{d}{d\lambda}g_{ab}(\lambda)+(g_{ab})^{-2}\frac{d^2}{d\lambda^2}g_{ab}(\lambda)\right)\lvert_{\lambda=0}$$
where
$$
\frac{d^2}{d\lambda^2}g_{ab}(\lambda) = \frac{d}{d \lambda} (h^{1}_{ab} + 2 \cdot \frac{1}{2!} \lambda h^2_{ab}) = h^2_{ab}.
$$
Therefore
$$\frac{d^2}{d\lambda^2}(g_{ab}(\lambda))^{-1}\lvert_{\lambda=0}
=-\left( -2\tilde{g}^{af}\tilde{g}^{bg}\tilde{g}^{cd}h^1_{fc}h^1_{dg}+\tilde{g}^{ac}\tilde{g}^{bd}h^{2}_{cd} \right)$$
$$=2h^{1ac}h^{1b}_c-h^{2ab}$$
so to construct the complete inverse metric up to the second order you need this generic form
$$g^{ab}(\lambda)=g^{ab}(0)+\frac{d}{d\lambda}(g^{ab}(\lambda))\lvert_{\lambda=0}+\frac{1}{2}\frac{d^2}{d\lambda^2}(g^{ab}(\lambda))\lvert_{\lambda=0}$$
plugging the quantities that we have already computed you obtain
$$g^{ab}(\lambda)=\tilde{g}^{ab}-\lambda h^{1ab}+\lambda^2(h^{1ac}h^{1b}_c-\frac{1}{2}h^{2ab})$$
a check that you should do to keep all in order is for example check the common delta relation with the total metric
$$g^{ac}(\lambda)g_{cb}(\lambda)=\delta^a_b$$
A: For your Minkowski background metric:
$$g_{\mu\nu}=\eta_{\mu\nu}+\kappa h_{\mu\nu}$$
We have that the perturbation can be written as:
$$\delta g_{\mu\nu}=g_{\mu\nu}-\eta_{\mu\nu}=\kappa h_{\mu\nu}$$
We also know that, at first order:
$$g^{\mu\nu}=\eta^{\mu\nu}-\kappa h^{\mu\nu}$$
Now we want to find it's covariant form, which goes like:
$$\delta g^{\mu\nu}=-g^{\mu\lambda}\delta g_{\lambda\rho}g^{\rho\nu}$$
Now simply substitute into this equation from our other equations:
$$=-\left(\eta^{\mu\lambda}-\kappa h^{\mu\lambda}\right)\left(\kappa h_{\lambda\rho}\right)\left(\eta^{\rho\nu}-\kappa h^{\rho\nu}\right)$$
Throwing away the third order term we obtain:
$$=-\kappa h^{\mu\nu}+\eta^{\mu\lambda}\kappa h_{\lambda\rho}\kappa h^{\rho\nu}+\kappa h^{\mu\lambda}\kappa h_{\lambda\rho}\eta^{\rho\nu}$$
$$=-\kappa h^{\mu\nu}+\kappa h_{\rho}^{\mu}\kappa h^{\rho\nu}+\kappa h^{\mu\lambda}\kappa h_{\lambda}^{\nu}\eta$$ Since the metric must be symmetric, so must the perturbation be also hence we can write:
$$\delta g^{\mu\nu}=-\kappa h^{\mu\nu}+2\kappa h_{\rho}^{\mu}\kappa h^{\rho\nu}$$
Now I got a factor of 2 different from your reference, Which I think can be eliminated by applying requirement for the total metric:
$$g^{\mu\nu}g_{\mu\nu}=\delta_{\mu}^{\mu}$$ But I think you get the Idea, it's a process that just grows outrageously in tediousness with each higher order. Cheers!! (:
