In what sense is the stress-energy tensor the derivative with respect to the metric? In Di Francesco et al (the big yellow book), section 2.5.2, it is suggested that the (symmetrized) stress energy tensor can be interpreted as the functional derivative of the action with respect to he metric.
I think I understand what is said in the book, and I interpret it as follows:
Let $\epsilon^{\sigma}\partial_{\sigma}$ be the vector field corresponding to an infinitesimal diffeomorphism of spacetime $\mathbb{R}^4$. Assume that $\epsilon^{\sigma}\partial_{\sigma}$ vanishes at ${\infty}$. (This assumption is not written in the book but it seems necessary for what follows).
This induces an infinitesimal transformation of the space of fields, (functions of the form $\varphi: \mathbb{R}^4\to \mathbb{R}$).
In turn, this induces a infinitesimal variation of the action, and a computation shows that
\begin{align}\delta S = -\frac{1}{2} \int d^4 x \,T^{\mu v} (\partial_{\mu} \epsilon_{\nu} + \partial_{\nu} \epsilon_{\mu})\tag{1}\end{align}
where $T_{\mu\nu}$ is the (symmetrized) stress energy tensor.
Then one notices that $-(\partial_{\mu} \epsilon_{\nu} + \partial_{\nu} \epsilon_{\mu})$ is exactly equal the variation of a flat metric tensor $\delta_{\mu\nu}$ under the infinitesimal action of $\epsilon^{\sigma} \partial_{\sigma}$:
\begin{equation}\delta g_{\mu \nu} = -(\partial_{\mu} \epsilon_{\nu} + \partial_{\nu} \epsilon_{\mu}).\end{equation}
Matching this to the previous equation, one can say that
\begin{align}T_{\mu\nu} =: -2 \frac{\delta S}{\delta g_{\mu\nu}}.\tag{2}\end{align}
My question is, how can we actually interpret the right hand side of this equation? I don't see how S depends on the metric at all. In the definition of the action, there is no mention of the metric
I see that a diffeomorphism induces a change in the field and hence a change in the action. But how does one implement an arbitrary change in metric $\delta g$? 
Does it correspond somehow to a change in the Lagrangian? For example, consider the simplest case of a one dimensional Lagrangian $$L= \frac12 m \dot{x}^2 + V(x)\tag{3}$$ (so we are working on $\mathbb{R}$ instead of $\mathbb{R}^4$.)
I know that the kinetic term of the Lagrangian is usually taken to be related to the metric, but how does the potential term $V(x)$ depend on the metric?
Or am I overthinking things, and is the equation $\ref{2}$ just suggestive language/notation?
 A: 
I don't see how S depends on the metric at all. In the definition of the action, there is no mention of the metric.

The metric comes in three places. First, the integration measure is actually $d^4x \, \sqrt{-g}$, not $d^4x$. Second, every time an index is raised or lowered, it is done using the metric. Third, all partial derivatives should be covariant derivatives, which depend on the metric. All these are suppressed in flat spacetime for convenience, but they're always there.
A: *

*For eq. (2) to be well-defined, the action should be general covariant. This means that the matter fields (which could have spin) should couple in a covariant fashion to a geometric background, which Ref. 1 assumes is a metric tensor $g_{\mu\nu}$. (If there are half-integer spin fields, one must couple to a vielbein instead.) 

*Ref. 1 shows in the analysis leading up to eq. (1) that the Belinfante-improved SEM tensor is equal to the metric/Hilbert SEM tensor (2). This is also the topic of this related Phys.SE post.

*As for OP's example (3), the Lagrangian is implicitly multiplied with a world-line einbein field to restore worldline reparametrization invariance, and there's implicitly a target space metric stuck in between the $\dot{x}$. (Moreover, the non-relativistic Lagrangian (3) can of course be replaced with a relativistic version.) 
References:


*

*P. Di Francesco, P. Mathieu and D. Senechal, CFT, 1997; Subsection 2.5.2.

