Electric field in a uniform time-varying magnetic field Suppose a homogenous magnetic field $\vec{B}$ in vacuum that varies with time, but always points in the $z$-direction. This induces a curl in the electric field $\vec{\nabla} \times \vec{E} = -\frac{\partial B}{\partial t} $, which is also uniform in space and points in the $z$-direction.
If we calculate the integral of this on a horizontal loop we get a non zero EMF through the loop, which means that the horizontal electric field must be non-zero at least in a portion of space. Because of translational symmetry one could argue that if $\vec{E} $ is non zero in one point, it must be non zero everywhere. Moreover, it should have the same value everywhere, which is an absurd, since this would mean the curl is zero everywhere, and so would be the EMF.
Where is the mistake in the argument?
Is it that a perfectly uniform time-varying magnetic field is inconsistent with Maxwell's equations? Or does it have something to do with Lorentz/Poincaré invariance being the proper symmetry of the system?
My first thought was that the field can't be uniform and dependent at the same time because it takes time for the change in the field to propagate, but I would like to have a more elaborate and/or mathematical answer, if this reasoning is correct.
 A: The main reason this argument is giving counter-intuitive results is, as the comments suggest, boundary conditions. Generally speaking, taking domains to be infinite in the study of differential equations gives rise to 'badly behaved' functions - discontinuities, Dirac deltas and other non-differentiable objects are readily obtained when differentiating functions that live on infinite domains. (A classic example is $\nabla^2 \frac{1}{r} = -4\pi \delta^3(\vec{r})$).
Essentially, the issue is that the solution of a differential equation is not really a function. They do not in general have the property that you can actually evaluate them at a point, so trying to think about what the solutions's value is  will always give confusing results. In this case, to solve your problem, we want an electric field such that $\nabla \times E = -f'(t)\hat{z}$, or rather
$$\partial_y E_x - \partial_x E_y = f'(t)$$
Firstly, note that the solution for $E$ is not unique - adding any irrotational field $\vec{F}$ to the electric field globally does not change this equation. One solution is $\vec{E}^{(1)} = \hat{x}yf'(t)$ - there's nothing wrong with that, but remember that $\vec{E}^{(2)} = -\hat{y}xf'(t)$. is just as good.
This is a bit of an issue, since electric fields can be directly measured by e.g. a test charge. In order to decide which of this infinite array of solutions is physical, you would need to specify a boundary condition on the electric field.
However, you can still get physical results without specifying the boundary conditions. Look at the integral form of Maxwell's equations, where all of the scalar divergences and Dirac deltas have been implicitly integrated out.
$$\oint_{\partial S} \vec{E} \cdot d\vec{l} = -\frac{\partial}{\partial t}\iint_S \vec{B} \cdot d\vec{A} = -f'(t) A_\perp$$
Where $A_\perp$ defines the cross-sectional area of the surface $S$ that is 'facing' the $z$ axis.
Then we have a clear result- The AC signal that a wire loop will pick up is a direct measure of the time derivative of $f$, amplified by the area $A_\perp$. This is the relevant physics.
None of these problems arise if you keep your system of charges and currents finite, such that the solutions you get are well-defined and make physical sense.
A: 
Because of translational symmetry one could argue that if $\vec{E} $ is non zero in one point, it must be non zero everywhere.

We can't, because we do not know anything about translation symmetry of electric field in this system. The system is not specified in detail enough. For example, we do not know where the charges are.
We only know magnetic field in detail. However, from Maxwell's equations we can infer some constraints on the electric field. For curl of electric field, we have
$$
\nabla \times \mathbf E  = - \frac{\partial \mathbf B}{\partial t}
$$
which is known, and for the rate of change of electric field (assuming there is no current anywhere), we have
$$
\frac{1}{c^2}\frac{\partial \mathbf E}{\partial t} = c^2 \nabla \times \mathbf B = 0
$$
since the magnetic field is uniform.
So electric field is constant in time and we know its curl is non-zero. But from this we can't derive $\mathbf E$ uniquely. There is infinity of valid electric fields obeying these conditions and these fields do not neccessarily have the translation symmetry mentioned.

Is it that a perfectly uniform time-varying magnetic field is inconsistent with Maxwell's Equations? Or does it have something to do with Lorentz/Poicarré Invariance being the proper symmetry of the system?

No and no.

My first thought was that the field can't be uniform and time-dependant at the same time because it takes time for the change in the field to propagate...

Mathematically, magnetic field can be both uniform and time-dependent. In reality, we do not know of such system and do not expect to ever find one - all magnetic field systems have field varying in space and decaying with distance from sources.
Infinite uniform magnetic field is a mathematical special case, not a realistic EM field.
