In the Lorentz transformation of the field $\partial_\mu\phi(x)$ (Peskin, p.36)
\begin{eqnarray} \partial_\mu\phi(x)\to\partial_\mu(\phi(\Lambda^{-1}x))=(\Lambda^{-1})^\nu_{\phantom{\nu}\mu}(\partial_\nu\phi)(\Lambda^{-1}x), \tag{3.3} \end{eqnarray}
I don't understand how to derive $(\Lambda^{-1})^\nu_{\phantom{\nu}\mu}$ in the r.h.s. and what is difference between $(\partial_\mu\phi)(x)$ and $\partial_\mu(\phi(x))$?
Let's forget about the indices for a while and do the chain rule: $$ \partial_x\left(f(\lambda^{-1} x)\right) =\dfrac{\partial f(\lambda^{-1} x)}{\partial x} =\lambda^{-1}\dfrac{\partial f(\lambda^{-1} x)}{\partial (\lambda^{-1} x)} \quad. $$ This explains the $\Lambda^{-1}$ prefactor on the RHS. If you understand this, adding indices should be pretty straightforward.
The second part of your question is caused by the standard abuse of notation. Again, let's suppress the indices for a moment. Since $$ \dfrac{\partial f(s)}{\partial s} \equiv (\partial_s f)(s) \quad, $$ we can write $$ \dfrac{\partial f(\lambda^{-1} x)}{\partial (\lambda^{-1} x)} = (\partial_s f)(s) \biggr\rvert_{s=\lambda^{-1}x} = f^{\,\prime}(\lambda^{-1}x) \quad. $$
$\partial_\mu(\phi(\Lambda^{-1}x))$ corresponds to the derivative of the function $\tilde{\phi}(x)=\phi(\Lambda^{-1}x)$ with respect to $x^\mu$: $$ \partial_\mu(\phi(\Lambda^{-1}x)) = \dfrac{\partial \tilde{\phi}(x)}{\partial x^\mu} = \dfrac{\partial \phi(\Lambda^{-1}x)}{\partial x^\mu} \quad. $$ $(\partial_\mu\phi)(\Lambda^{-1}x)$ corresponds to calculating the derivative of $\phi(s)$ with respect to $s$, and then evaluating this derivative at the point $s = \Lambda^{-1}x$: $$ (\partial_\mu\phi)(\Lambda^{-1}x) = \left.\dfrac{\partial\phi(s)}{\partial x^\mu}\right|_{s=\Lambda^{-1}x} \quad. $$
