Interpretation of Renormalization Group For the purposes of this question, I will be talking about systems in statistical mechanics (e.g the Ising Model) so I will assume the system of interest has a natural cutoff frequency $\Lambda$.
For systems in statistical mechanics, one can coarse-grain the system to obtain the following expression for the partition function:
$$Z = \int D\phi e^{-\beta F[\phi]},$$
where $F[\phi]$ is a functional of the form $$F[\phi] = \int d^dx \left[ \frac{1}{2} \gamma\nabla\phi \cdot \nabla\phi +\mu\phi^2 + g \phi^4 + ...\right].$$ One can then use Wilsonian Renormalization Group procedure to identify relevant and irrelevant interactions in the long range physics. More specifically, one can integrate over high momentum modes ($\Lambda'<k<\Lambda$) to create an effective theory for the long distance. This can be done by using the formula
$$Z = \int \prod_{k < \Lambda'}d\phi^-_{k}e^{-F[\phi^-_{k}]} \int \prod_{\Lambda'<k < \Lambda}d\phi^+_{k}e^{-F[\phi^+_{k}]}e^{-F_I[\phi^+_{k}, \phi^-_{k}]} \tag{2},$$
where $\phi^-_{k}$ and $\phi^+_{k}$ corresponds to the low and high energy modes respectively and $F_I[\phi^+_{k}, \phi^-_{k}]$ is the interaction energy resulting from separating the field into high and low energy modes.
My question involves the fact that we can use RG to obtain a long range description of the system. Let me expand:
Imagine we start with a free energy $F_{0}$ that contains all the possible microscopic interactions. I shall denote the coupling constants by the vector $\textbf{K}$. The partition function $Z$ contains the complete information about the theory. Now we consider an RG transformation in momentum space where we integrate over large momentum modes, rescale the fields and the momenta. Furthermore, also suppose we computed the integral given in equation (2) exactly without resulting to perturbation theory. This RG transformation induces a flow in the parameter space where $\textbf{K} \rightarrow R(\textbf{K})$. Some terms, due to scaling, will get suppressed as they are irrelevant. However, note that the partition function, is still intact as we have done everything perfectly without resulting to perturbation. Imagine we continue this perfect RG transformation until the irrelevant interactions are virtually zero. In this case, we have reached a "long-distance" physics as we have discarded the microscopic interactions (their effects are included as modifications to the coupling constants for the relevant terms). However, note that throughout this process, the partition function, which contained all of the information about the high energy physics, hasn't changed at all. Therefore, this must mean, the effective long range theory should also explain the short range physics as well, which was supposedly discarded. Here, I believe we are at a contradiction and I cannot see where I went wrong. Help would be very much appreciated.
 A: In $D$ spacetime dimensions, the Wilson effective action at scale $\Lambda$ is an infinite series expansion
\begin{equation}
S_{\rm eff}[\Phi; \Lambda] = \int d^D x \sum_n \frac{c_n}{\Lambda^{D-\Delta_n}} \mathcal{O}_n(\Phi, \partial \Phi, \partial^2 \Phi, \cdots)
\end{equation}
where $\Delta_n$ is the dimension of the operator $\mathcal{O}_n$.
Now, you are correct, that if we:

*

*knew all the $c_n$ exactly,

*knew how to do calculations in quantum field theory with an arbitrarily complicated Lagrangian,

*knew how to do calculations in quantum field theory where all the operators in this expansion were equally important,

then we could use the effective action at $\Lambda$ to compute observables at arbitrarily high energies, just like you said.
However, in practice, none of the above three assumptions are ever true.

*

*We only ever have guesses or finite experimental precision for the $c_n$ (except maybe in special theories of interest to mathematical physicists with a lot of symmetry, or some toy examples). There can be massive particles above the cutoff of the effective theory that have never been produced in a collider, that are important at high energies, but only have tiny effects on the $c_n$ at low energies that are practically impossible to measure.

*Perturbation theory is a powerful tool, and we can only really go beyond it in very special theories with a lot of symmetry. There will be an infinite number of interaction terms we have to take into account here, which will lead (in practice) to impractically complicated calculations.

*Again going with the perturbation theme, the typical starting point of perturbation is that we are close to a free theory. Not only will we have an infinite number of interaction terms, but they will all be at least as important as the quadratic terms, or maybe even more important. At this point, we really have no idea (in general) how to treat such a theory. There are only results known in special cases, for example with a lot of symmetry or small $D$.

The beauty of the RG is precisely that it explains why we can use our ordinary understanding of renormalizable QFTs, at low energies, without having to know all the details of high energies. However, the double edged sword here, is that it becomes very difficult (impossible, really) in practice to reverse engineer what's going on at high energies in the way you are suggesting. While that is possible in principle, in practice it requires much, much more experimental precision and calculational skill than we possess at the present time.
