Your main problem is between your first and second equation. You are indeed correct that outside of the solenoid, the curl of the electric field is zero. However, this is not enough to conclude that
$$\oint_{\partial\Sigma}\textbf{E}\cdot\mathrm{d}\boldsymbol{\ell}=0$$
if $\partial\Sigma$ is a loop which goes around the solenoid.
This is a little counter-intuitive if you've always had "$\boldsymbol{\nabla}\times\textbf{E}=0$ implies that $\textbf{E}$ is conservative" drilled into your head. Indeed, because $\boldsymbol{\nabla}\times\textbf{E}=0$ in some open neighbourhoods of space, but not globally (in particular, the curl fails to vanish inside the solenoid), then you will only be able to find a potential for $\textbf{E}$ such that $\textbf{E}=-\boldsymbol{\nabla}\Phi$ locally. You will not be able to find such a function globally and, in particular, you will not be able to find such a function on an open neighbourhood that surrounds but does not include the solenoid itself.
The easy way to see this is just through direct application of Stoke's theorem. We have
$$\oint_{\partial\Sigma}\textbf{E}\cdot\mathrm{d}\boldsymbol{\ell}=\int_{\Sigma}(\boldsymbol{\nabla}\times\textbf{E})\cdot\mathrm{d}\textbf{S}.$$
Thus, if $\partial\Sigma$ doesn't surround the solenoid (or if it can be continuously deformed into a loop that doesn't surround the solenoid without passing through the solenoid), then this integral vanishes, and $\textbf{E}$ is conservative locally. However, if $\partial\Sigma$ does surround the solenoid, then the surface integral picks up contributions for which the curl of $\textbf{E}$ doesn't vanish, and the integral is no longer nonzero.
I could stop here, and it'd probably be fine, but this answer leaves something to be desired. So I'll conclude by mentioning the actual origin of this problem -- namely topology. By using this setup, what we have essentially done is claimed that $\boldsymbol{\nabla}\times\textbf{E}=0$ everywhere except for at the location of the solenoid. Thus, let's just remove the solenoid from space and talk about what happens. If our space $X$ now takes the form of $\mathbb{R}^3$ with a cylinder removed, then the problem now becomes:
If $\boldsymbol{\nabla}\times\textbf{E}=0$ everywhere on $X$, then does the induced EMF around any loop in $X$ vanish?
Or, equivalently (and more mathematically), we have
If $\boldsymbol{\nabla}\times\textbf{E}=0$ everywhere on $X$, then does there exist a function $\Phi$ such that $\textbf{E}=-\boldsymbol{\nabla}\Phi$ everywhere on $X$?
The latter form of this question is a well-known topological question with a well-known answer. In particular, the answer depends on a mathematical invariant known as the de-Rham cohomology of the space $X$, which is a group that encodes certain topological properties of the space. If the de-Rham cohomology group is trivial, then $\boldsymbol{\nabla}\times\textbf{E}=0$ everywhere implies that $\textbf{E}$ is conservative everywhere. Otherwise, this is simply not the case. In our example, the (first) de-Rham cohomology group of $\mathbb{R}^3$ minus a cylinder is nontrivial (I believe it is the integers $\mathbb{Z}$, but someone correct me if I'm wrong), and thus the vanishing of the curl outside of the solenoid isn't enough to guarantee that $\textbf{E}$ is conservative globally.
These types of topological arguments are the origin of several physical effects such as the Aharonov-Bohm effect, the Dirac quantization of magnetic and electric charge, and the analogous quantization of charges in extended objects (branes) in string theory/M-theory.
I hope this helps, and I hope I've given enough information for you to start learning more about this stuff!
