How does an operator on the form $e^{\beta \hat{H}}$ act on a eigenstate? The notion of operators in exponentials is a bit confusing to me. I know that that in some cases one can use the Taylor series of $e^x$, but how do you work with them when that's not the case? 
 A: $\newcommand{\ket}[1]{\left| #1 \right>}$Let's take a step back and look at an arbitrary linear operator $H$ (I don't like putting hats on operators). You probably know from your linear algebra classes how to act on a vector $\ket \psi$. Well it is just $H \ket \psi = \ket \varphi$, which is of course another vector in your vector space, which I have called $\ket \varphi$. Now you probably also know, how $H^2$ acts on a vector; namely you just act with the operator $H$ twice:
$$H^2 \ket \psi = H (H \ket \psi) = H \ket \varphi =  \ket \chi$$
where I defined the vector $\ket \chi := H \ket \varphi $
By induction, you also know how $H^n$ acts for $n \in \mathbb N$. 
Remember also that you are in a vector space that is you can add two vectors, and in particular two linear operators. For example, let's look at the operator $H^2+H$:
$$ (H^2+H) \ket \psi = H^2 \ket \psi + H \ket \psi = \ket \chi + \ket \varphi   $$
Again by induction you know for a polynomial $p(x) = \sum_{k=1}^n a_k x^k$, where $a_k \in \mathbb C$ some constant coefficients, how the operator $p(H)$ acts on states:
$$ p(H) \ket \psi = \sum_{k=1}^n a_k (H^k \ket \psi) $$
Without worrying too much about convergence issues (we are physicists (!)), we can define the exponential map of an operator:
$$  \exp(H) = \sum_{n \in \mathbb N} \frac{1}{n!} \, H^n \implies \exp(H) \ket \psi = \sum_{n \in \mathbb N} \frac{1}{n!} \, (H^n  \ket \psi) $$
The upshot is that you have to define the exponential of an operator and there is no way around it. There is however a special case, when for example you have an eigenvector of the operator. Let's assume that the vector $\ket E$ is an eigenvector of $H$ with eigenvalue $E$ i.e. $H \ket E = E \ket E$, then we have:
$$ \exp(H) \ket E =  \sum_{n \in \mathbb N} \frac{1}{n!} \, (H^n  \ket E ) =  \sum_{n \in \mathbb N} \frac{E^n}{n!}\ket E  = \exp(E)\ket E$$
In analogy, you can convince yourself that if you can write a function $f$ in taylor series and if $\ket E$ is an eigenvector of $H$ as above then we have $f(H) \ket E = f(E) \ket E $.
I hope that this helps you understand the exponential of an operator. If you have question, don't hesitate to ask!
A: The exponential of an operator is defined by its series expansion:
\begin{align}
e^{\beta \hat H}&= \hat 1 +\beta \hat H+\frac{1}{2}\beta^2\hat H^2+\ldots  \, ,\\
&=\sum_{k=0}^\infty \frac{\beta^k \hat H^k}{k!}\, .\tag{1}
\end{align}
If $\vert\psi\rangle$ is an eigenstate of $\hat H$ so that
$$
\hat H\vert\psi\rangle=\lambda\vert\psi\rangle\, ,\quad 
\hat H^2\vert\psi\rangle=\lambda^2\vert\psi\rangle\, ,\quad
\hat H^n\vert\psi\rangle=\lambda^n\vert\psi\rangle
$$
then
$$
e^{\beta \hat H}\vert\psi\rangle
=\sum_{k=0}^\infty \frac{\beta^k \hat H^k}{k!}\vert\psi\rangle
=\sum_{k=0}^\infty \frac{\beta^k \lambda^k}{k!}\vert\psi\rangle
=e^{\beta \lambda}\vert\psi\rangle\, .
$$
If $\vert\psi\rangle$ is not an eigenstate, there are fewer options.  First, work out in details
$$
e^{\beta \hat H}\vert\psi\rangle=\sum_{k=0}^\infty \frac{\beta^k \hat H^k}{k!}\vert\psi\rangle \tag{2}
$$
and hope that each $\hat H^k\vert\psi\rangle$ can be simplified.  This occurs, for instance, with Pauli matrices where - say - $\hat H=\sigma_j$ and $\sigma_j^2=\hat 1$ for any $j=x,y,z$.   It may then be possible to resum the series in  (2).
The second alternative is to expand $\vert\psi\rangle$ in terms of eigenstates of $\hat H$.  Thus, if $\vert\mu_\alpha\rangle$ is such that 
\begin{align}
\hat H\vert\mu_\alpha\rangle&=\lambda_\alpha\vert\mu_\alpha\rangle\, ,\\
\vert\psi\rangle&=\sum_{\alpha}\vert\mu_\alpha\rangle\langle \mu_\alpha\vert\psi\rangle
\end{align}
then 
$$
e^{\beta \hat H}\vert\psi\rangle=\sum_{\alpha}e^{\beta\hat H}\vert\mu_\alpha\rangle\langle \mu_\alpha\vert\psi\rangle
=\sum_{\alpha}e^{\beta\lambda_\alpha}\vert\mu_\alpha\rangle\langle \mu_\alpha\vert\psi\rangle
$$
which cannot be resummed further in general. 
You can still calculate, for instance,
\begin{align}
\langle\psi\vert e^{\beta \hat H}\vert\psi\rangle &=
\sum_{\alpha\kappa}e^{\beta \lambda_\alpha} \langle\psi\vert\mu_\kappa\rangle
\langle \mu_\kappa\vert\mu _\alpha\rangle\langle \mu_\alpha\vert\psi\rangle\, ,\\
&=
\sum_{\alpha}e^{\beta \lambda_\alpha} \langle\psi\vert\mu_\alpha\rangle
\langle \mu_\alpha\vert\psi\rangle 
\end{align}
where $\langle \mu_\kappa\vert\mu _\alpha\rangle=\delta_{\mu\alpha}$ has been used.
A: The definition of the matrix exponential comes straight from the Taylor expansion of $e^x$. Thus, you can always substitute the matrix exponential for its definition.
You can still use the Taylor expansion in the example you provided. You just take $x$ to be $\beta \hat{H}$ and apply the Taylor expansion as you would normally.
