What is the general relativity explanation for why objects at the center of the Earth are weightless? The idea that as you move through the earth you get a symmetric cancelling of gravitational acceleration which approaches zero acceleration due to gravity at the center of the earth makes a lot of sense I think. I also think that is a pretty easy idea to understand physically.
On the other hand, I am just beginning to study and understand the general relativistic explanation for gravity and I'm wondering how this same idea is explained using general relativity. That is, what is the explanation for why gravitational acceleration decreases (linearly if I'm not mistaken) as you approach the center of the earth using general relativity? Specifically, I would like to understand what is going on with the curvature of spacetime in this scenario? Is it becoming less curved? Some math would be appreciated in answers!
 A: In this answer we assume a spherically symmetric spacetime, no cosmological constant $\Lambda=0$, and signature convention $(-,+,+,+)$ for the metric. 
I) Birkhoff's theorem (BT) only works for a vacuum branch of a spherically symmetric spacetime, i.e. in a radial interval $r_1<r<r_2$ without any matter, cf. e.g. this Phys.SE post. Therefore BT would apply to a hollow planet, cf. e.g. this Phys.SE post. Most importantly,
the Newtonian shell statement
$$ \text{A spherical shell does not contribute to the gravity}$$
$$\tag{1}\text{experienced by an object within it.} $$
(borrowed from John Davis's answer) is only also valid in GR if there is no pressure $p=0$ between the shell and the interior. (Statement (1) holds in Newtonian gravity, due to Newton's shell theorem. See also this Phys.SE post.)
II) However, here we are interested in a massive planet with pressure $p>0$, of radius $R$. As we shall see, in such situations we can not use BT.
To proceed, we should first and foremost introduce a non-zero stress-energy-momentum (SEM) tensor $T^{\mu\nu}$ for the planet's matter, which acts as a source term in the EFE. For a static, spherically symmetric planet, the EFE leads to the Tolman-Oppenheimer-Volkoff equation. Let us for simplicity in this answer furthermore assume that the system is a perfect fluid of uniform density $\rho_0$. The SEM tensor then reads
$$\tag{2}  T^{\mu\nu}~=~ \left(\rho_0 + \frac{p(r)}{c^2}\right)U^{\mu}U^{\nu} +p(r)g^{\mu\nu}.$$
The corresponding metric tensor was already found by Schwarzschild in 1916, cf. Ref. 1-3 and John Rennie's answer. The metric is of the form
$$\tag{3} ds^2 ~=~g_{\mu\nu}~dx^{\mu}~dx^{\nu} ~=~ - \exp\left[ \frac{2\Phi(r)}{c^2}\right] (c~dt)^2 + \frac{(dr)^2}{h(r)}+r^2d\Omega^2, \quad r~<~R.\qquad$$
In the $g_{rr}$-component of the metric (3), the structure function
$$\tag{4} h(r)~:=~ 1-\frac{r_S}{R^3} r^2~=~1-\frac{a}{c^2} r^2, \qquad a~:=~\frac{8\pi G\rho_0}{3} ,$$
has a quadratic dependence on the radial coordinate $r$, where 
$$\tag{5} r_S ~:=~ \frac{2GM}{c^2}, \qquad M~:=~\frac{4\pi}{3}R^3\rho_0,$$
is the Schwarzschild radius. To ensure that $h(r)>0$, we must impose $r_S/R <1$, i.e. the planet is not a black hole.
III) The corresponding pressure is
$$ \tag{6} p(r)~=~ \rho_0c^2 \frac{\sqrt{h(r)} -\sqrt{h(R)}}{3\sqrt{h(R)} -\sqrt{h(r)}} . $$
In order for the denominator of the pressure profile (6) to stay positive, we must impose the interesting inequality
$$  \tag{7} \frac{r_S}{R} ~<~\frac{8}{9} \qquad\Leftrightarrow\qquad \forall r~\in~[0,R]:~ 9h(R)~>~h(r)     . $$
IV) In the $g_{tt}$-component of the metric (3), the structure function 
$$ \Phi(r)~=~c^2\ln\left(\frac{3}{2} \sqrt{h(R)} - \frac{1}{2} \sqrt{h(r)}\right) $$
$$ \tag{8}~=~ \frac{a}{4}\left(r^2-3R^2\right) +\frac{a^2}{32c^2}\left(r^4+6r^2R^2-15R^4\right) +{\cal O}(c^{-4})$$
becomes the Newtonian potential in the Newtonian limit $c\to \infty$, cf. e.g this Phys.SE post. (The corresponding electrostatic potential is given in my Phys.SE answer here.)
V) As explained in John Rennie's answer, the acceleration is governed by the Christoffel symbol
$$ \tag{9} \Gamma^r_{tt}~=~-\frac{1}{2}g^{rr} \partial_r g_{tt}
~=~ \frac{r_S r}{4R^3} \left(3\sqrt{h(R)}\sqrt{h(r)} -h(r) \right),$$ 
which vanishes at the center $r=0$, cf. OP's title question 
$$ \tag{10} \text{Why are objects at the center of the Earth weightless?}$$
Actually, the title question (10) follows from just the spherical symmetry alone, independently of the underlying theory: An acceleration $3$-vector at the center $r=0$ breaks spherical symmetry unless it is zero! 
VI) Returning to the statement (1), the structure function $h(r)$ in eq. (4) does indeed obey the philosophy of statement (1), but $h(R)$ does not. For a given radial coordinate $r<R$, the function $h(R)$ also refers to the mass-parts beyond $r$. Looking at the non-trivial dependence of $h(R)$ in eqs. (6), (8), and (9), we conclude that the statement (1) is not fulfilled in this case. 
Let us compare with Newtonian gravity. In Newtonian gravity, the potential $\Phi(r)$ does depend on the mass-distribution beyond $r$, but only via an additive constant. (The additive constant is adjusted so that $\Phi(r)=0\Leftrightarrow r=\infty$ because we assume an asymptotically flat spacetime.) So if we differentiate $\Phi(r)$ and consider the gravitational acceleration $g(r)$ instead, it will not depend on the mass-distribution beyond $r$, and hence obey statement (1). 
TL;DR: The statement (1) is no longer true in GR if there is pressure $p>0$ present.  
References:


*

*K. Schwarzschild, Über das Gravitationsfeld einer Kugel aus inkompressibler Flüssigkeit nach der Einstein’schen Theorie, Sitzungsberichte der Königlich-Preussischen Akademie der Wissenschaften (1916) 424.

*MTW; Section 23.7.

*R. Wald, GR, 1984; Section 6.2.
A: The geometry of spacetime is described by a function called the metric tensor. If you're starting to learn GR then any moment you'll encounter the Schwarzschild metric that describes the geometry outside a sphrically symmetric body. When you go inside the body the geometry is described by the (less well known) Schwarzschild interior metric.
The exact form of the interior metric depends on how the density of the spherical body changes with depth, so for real objects is has a rather complicated form. However for a body with constant density it simplifies a bit and looks like:
$$ ds^2 = -\left[\frac{3}{2}\sqrt{1-\frac{2M}{R}} - \frac{1}{2}\sqrt{1-\frac{2Mr^2}{R^3}}\right]^2dt^2 + \frac{dr^2}{\left(1-\frac{2Mr^2}{R^3}\right)} + r^2 d\Omega^2 \tag{1} $$
In this equation $r$ is the distance from the centre of the object with mass $M$, and $R$ is the radius of the object. So this metric applies for $r \le R$.
If you ignore relativity for a moment and consider just Newtonian gravity, then the radial acceleration of an object near a spherical body is:
$$ a = \frac{d^2r}{dt^2} = -\frac{GM}{r^2} $$
In GR the situation is rather more complicated (which I'm sure comes as no surprise) but we can define an analogous quantity called the four-acceleration. In particular we want the radial component $d^2r/d\tau^2$. Although this looks like the Newtonian acceleration above, it's the second derivative with respect to proper time $\tau$ not the coordinate time $t$.
We get the four acceleration using the geodesic equation:
$$ {d^2 x^\mu \over d\tau^2} = - \Gamma^\mu_{\alpha\beta} {dx^\alpha \over d\tau} {dx^\beta \over d\tau} $$
where the $\Gamma^\mu_{\alpha\beta}$ are called Christoffel symbols (of the second kind) and they depend on the spacetime curvature. This all looks rather complicated, but for spherically symmetric objects all but one of the sixteen terms on the right hand side are zero and the radial four acceleration of a stationary object is simply:
$$ \frac{d^2r}{d\tau^2} = - \Gamma^r_{tt} u^t u^t \tag{2} $$
This is the equation that tells us how the (four) acceleration varies with depth, and we can use it to show that you are weightless at the centre of the object. Equation (2) is going to go to zero if either $u^t$ becomes zero or $\Gamma^r_{tt}$ becomes zero. The quantity $u^t$ is the time component of the four-velocity, which you can think of as the time dilation factor. This is simply $dt/d\tau$ for constant $r$, $\theta$ and $\phi$, and we get it from the metric (1) simply by setting $dr = d\theta = d\phi = 0$ to give:
$$ ds^2 = -\left[\frac{3}{2}\sqrt{1-\frac{2M}{R}} - \frac{1}{2}\sqrt{1-\frac{2Mr^2}{R^3}}\right]^2dt^2 $$
Since $ds^2 = -d\tau^2$ a simple rearrangement gives:
$$ u^t = \frac{dt}{d\tau} = \frac{1}{\frac{3}{2}\sqrt{1-\frac{2M}{R}} - \frac{1}{2}\sqrt{1-\frac{2Mr^2}{R^3}}} $$
Let's just note that this does not go to zero as $r \rightarrow 0$, and move swiftly on to look at the other term $\Gamma^r_{tt}$. Calculating this involves some painful algebra, but Mathematica is good at this sort of thing and Danu helpfully used Mathematica to do the calculation for me. The result is:
$$\Gamma_{tt}^r= \frac{r}{2R^6}\left[2M^2r^2+MR^3\left(3\sqrt{1-\frac{2Mr^2}{R^3}}\sqrt{1-\frac{2M}{R}}-1\right)\right] $$
Yet another fiendishly complicated expression, but note that the whole thing is multiplied by $r/2R^6$ and that means if $r = 0$ the whole complicated expression is just zero.
And there's our result!
When $r = 0$ the Christoffel symbol $\Gamma_{tt}^r$ is zero and that means the radial four-acceleration is zero and that means you're weightless.
A: The explanation is Birkhoff's theorem, which states that the Schwarzschild solution is the unique spherically symmetric vacuum solution in general relativity. An immediate result of this is that, just as in Newtonian gravity, a spherical shell does not contribute to the gravity experienced by an object within it. If this were otherwise it would suggest the existence of non-Schwarzschild spherically symmetric vacuum solutions.
If we consider the Earth as a series of concentric spherically-symmetric shells, then for an object falling towards the centre you need only consider the mass contained contained within the sphere with a radius that is the same as the distance of the object to the centre.
The second derivative of the radial Schwarzschild coordinate of a radially infalling object with respect to its proper time is:
$$\frac{d^2r}{d\tau ^2} = -\frac{GM}{r^2}$$
If the density of the Earth is constant, then inside the Earth the mass contained within any shell is: 
$$M = \frac{4}{3} \rho \pi r^3 $$
Therefore for an object inside the Earth:
$$\frac{d^2r}{d\tau ^2} = -\frac{4}{3} G\rho \pi r$$
And the 'pull' towards the centre it experinces decreases linearly with distance to the centre.
Viewing spacetime as more or less curved at a point can be problematic, but in a certain sense the curvature of spacetime does become less curved for an observer inside the Earth as they approach the centre.
A: 
What is the general relativity explanation for why objects at the center of the earth are weightless?

At that location spacetime is locally flat. 
See the Wikipedia Riemann curvature tensor article and look at the schematic on the right:  

CCASA image by Johnstone, see Wikipedia 
Let's imagine we could take away the Earth and look more closely. Underneath the Earth, there's a small portion of spacetime that doesn't curve upwards. You can see it better on this plot of (Newtonian) gravitational potential on Wikipedia:    

CCASA image by AllenMcC, see Wikipedia

The idea that as you move through the earth you get a symmetric cancelling of gravitational acceleration which approaches zero acceleration due to gravity at the center of the earth makes a lot of sense I think. I also think that is a pretty easy idea to understand physically.

I don't think anybody would argue with that. 

On the other hand, I am just beginning to study and understand the general relativistic explanation for gravity and I'm wondering how this same idea is explained using general relativity. That is, what is the explanation for why gravitational acceleration decreases (linearly if I'm not mistaken) as you approach the center of the earth using general relativity? 

It isn't really any different to the non-relativistic explanation. A concentration of energy in the guise of a massive body conditions the surrounding space, altering its metrical properties, which are modelled as curved spacetime. See Einstein talking about it in the digital papers. Then if you're between two massive bodies, this effect is the same on either side, so there's no net effect. Being at the centre of the Earth is like being surrounded by lots of massive bodies. 

Specifically, I would like to understand what is going on with the curvature of spacetime in this scenario? Is it becoming less curved? Some math would be appreciated in answers!

I suspect the underlying issue here is an ambiguity concerning flat as in level, and flat as in not curved. Look at the plot. The curvature you can see is curved spacetime. By and large the curvature increases as you approach the centre, which is why people talk about "the strong curvature regime" in the context of black holes. Follow the plot and note the inflection points, and you can see that increases then decreases then increases. However it's very important to appreciate that this curvature relates to the tidal force. The slope of the plot relates to the force of gravity. The force of gravity depends on the "spacetime tilt" at that location, not on the curvature. See the tilted light cones on this Stanford article. At the centre of the Earth, there's a small part of the plot where there is no slope. At that location spacetime is locally flat and level, so there's no gravity. 
Note thought that if there's no curvature at all in your plot, it can't go from flat-and-horizontal in the centre to tilted. Hence curved spacetime is the "defining feature" of a gravitational field.      
