[Note: There are already some nice answers here, but they focus on the case of non-interacting fermions. While that is certainly an interesting case, it is quite a small fraction of the landscape of topological phases of matter, and some of its key features (in particular, the topological space being a vector bundle defined by the energy eigenstates) do not generalize to the more general case. Let me thus offer a different perspective, which I think covers all known cases of topological phases of matter (at least in the sense that this term is conventionally used).]
The defining property of topological phases of matter (be they non-interacting, or symmetry-protected, or intrinsically topologically ordered) is that their universal description only relies on topological information of the spacetime manifold on which they live (that is to say, it does not depend on the metric).
To unpack this, let me first explain what I mean by 'universal'. Technically, this refers to the properties which remain when one performs a renormalization group flow, which gets rid of short-distance physics. Practically, one can think of these as the properties which can be probed at long distances. The reason these universal properties are of interest is as the name suggests: they are independent of any specific realization of the phase. More precisely, two states of matter in the same phase will have the same universal properties. Indeed, suppose one has a path in parameter space where along each point the universal part only relies on topological information (we say that the low-energy theory is described by a topological quantum field theory (TQFT)): this implies that we did not encounter a quantum critical point along this path, since at a quantum critical point we have massless particles, implying a non-trivial stress-energy tensor which couples to the metric and is thus not topological (remember that $T^{\mu,\nu} \propto \frac{\delta S}{\delta g_{\mu,\nu}}$).
As already said, 'topological' means that these universal properties do not depend on the metric. As just discussed, this excludes the case of gapless fields. Conversely, does any gapped theory flow to a TQFT under RG? It is tempting to think so: if local excitations have some nonzero mass, then under RG their mass scale will increase, eventually flowing to infinite-mass. In this RG fixed point limit, no local degrees of freedom (d.o.f.) remain; the only remaining d.o.f. would thus have to be topological (we will discuss examples shortly). For this reason, for a long time, the study of 'topological phases of matter' was seen to be equivalent to the study of gapped (quantum) phases of matter. Now we know this is not the full story: there is something known as fracton order, which are gapped phases of matter which are not topological in the aforementioned sense. The loophole is quite subtle and not yet fully understood: it turns out that the usual notion of RG does not quite apply to these phases, and its resolution is an active field of research. (I will not discuss fractons further in this post, as they go beyond the conventional notion of topological order.)
So what answer does this suggest to your question? We see that, arguably, the 'topological space' relevant to topological phases of matter is spacetime itself. Of course in principle spacetime has a lot more structure than just a topological space, but the point of topological phases of matter---which are described by a TQFT---is that they do not care about any additional structure of spacetime. Note that this answer implies we cannot quite use the 'answer prompt' that you suggested: that prompt implicitly presumed that the topological space in question was encoding the information of the phase of matter. I am referring to your sentence: ''Topological properties of this space, such as ______, can correspond to physical properties''. This is indeed the case for free-fermion phases of matter, where it is possible to think of the vector bundle of energy eigenstates as being a topological space encoding the information of whatever phase you are in, with e.g. the Chern number of that vector bundle encoding physical properties such as Hall currents (see J. Murray's nice answer on this perspective). In a sense, this is a curiosity of free-fermion systems. More generally, we should not consider topological phases as being characterized by a topological space, but rather a topological map. To say it differently: for a topological quantum field theory, the (topological) spacetime manifold is part of the input, not the output. I.e., once you fix your TQFT, it can assign interesting invariants to whatever spacetime manifold you choose to put it on. E.g., for toric code topological order (which is a particular topological phase of matter popularized by A. Kitaev [although many people contributed to its discovery, including X.G. Wen, S. Sachdev and N. Read]), its low-energy theory is a topological map which takes as input an arbitrary closed spacetime manifold $X$ and as its output it gives a stable ground state degeneracy given by $4^g$ where $g$ is the genus of $X$ (this is just an example, ground state degeneracy is not the only quantity to look at).
This way of viewing topological phases of matter as topological maps can indeed be formalized by defining TQFTs as 'functors' (essentially: maps) from the category of cobordisms (essentially: spacetime manifolds with spatial boundaries) to the category of vector spaces (essentially: the Hilbert space of your physical states). I will not delve into this abstract notion here. Instead, I will go through the three main instances of topological phases of matter (i.e., (1) free-fermion topological insulators as explored in the previous posts here, albeit in a different jacket, (2) interacting symmetry-protected topological phases, and (3) intrinsic topological order) and give some examples of the TQFTs that describe them.
Example: topological insulators
A typical example of a topological free-fermion phase of matter is a 2D Chern insulator. Instead of giving an explicit lattice model (see e.g. these notes), I will discuss the effective TQFT that characterizes this phase of matter. To define this, we imagine coupling our system to an external gauge field $A$. Then its low-energy theory is claimed to be described by the following action:
$$ S_\textrm{CS}[A] = \frac{k}{4\pi} \int AdA = \frac{k}{4\pi} \int \varepsilon^{ijk} A_i \partial_j A_k \mathrm d x \mathrm d y \mathrm d t. $$
This action is well-defined on an arbitrary (orientable closed) manifold without having to appeal to a metric; this makes the action topological. This particular topological action is known as the Chern-Simons action. Moreover, one can argue that gauge-invariance implies that $k \in \mathbb Z$ (see e.g. Dunne's notes). This integer is a quantized topological invariant that characterizes this phase of matter.
When we say that 'this action describes our phase of matter', it means that all the salient features of the phase can be deduced from this action. E.g., the current can be obtained as $j^\mu = \frac{\partial S}{\partial A_\mu} = \frac{k}{2\pi} \varepsilon^{\mu \nu \lambda} \partial_\nu A_\lambda$. In particular, this tells us that the current in the $x$-direction is proportional to the electric field in the $y$-direction: i.e., we derive the quantum Hall effect (with the quantization of $k$ implying the quantization of the Hall conductance). Also other physical features, such as chiral edge modes, can be deduced from this action (see the aforementioned notes by Dunne for more on this). The fact that all these physical features are wrapped up in this topological action implies that these are robust features of a topological phase of matter. (In fact, one neat consequence is that it shows that these properties are stable upon adding interactions to the Chern insulator, since this effective action does not explicitly use the non-interacting nature of the underlying system.)
All good and well: a Chern insulator is apparently described by this topological action. But who ordered that? I.e., where does it come from? One practical answer is a phenomenological one, where one simply mutters 'if the shoe fits...'. By this I mean: we have found a topological action which reproduces the physical features which we know that Chern insulators have, hence it must be the correct action. However, this answer is perhaps not very satisfactory. Fortunately, there is a more systematic way to arrive at this action. Firstly, one starts from the underlying fermionic Hamiltonian $\hat H$ (e.g., a particular lattice model realizing a Chern insulator). This defines a partition function:
$$ Z = \textrm{tr} e^{-\beta \hat H}. $$
Using the usual tricks, one can rewrite the fermionic lattice operators in terms of Grassmannian field operators to essentially have something of the form
$$ Z = \int e^{-S[\psi]} \mathcal D[\psi]. $$
Using minimal coupling, one can couple this to an external gauge field. 'External' means that we do not integrate over it in the partition function:
$$ Z[A] = \int e^{-S[\psi,A]} \mathcal D[\psi] .$$
Now imagine explicitly performing the integral over the fermionic degrees of freedom. In the non-interacting case, this can be done explicitly since everything is quadratic in the fermions. One writes the result as
$$ Z[A] = e^{-S_\textrm{eff}[A]}.$$
While doing this integration is doable, it has some subtleties, as discussed extensively and explicitly in this paper by Witten and Yonekura. The end result turns out to be the Chern-Simons action!
What determines the integer $k\in \mathbb Z$ of the resulting Chern-Simons action? In fact, it turns out to be equivalent to the Chern number of the vector bundle of single-particle energy eigenstates in momentum space! This makes a link with the approach discussed in J. Murray's post. This non-trivial equivalence between the level $k$ and the Chern number is discussed pedagogically in Section 2.5 of these lecture notes by Witten.
For more on this field-theoretic perspective on topological insulators, see the seminal paper by Xiao-Liang Qi, Taylor Hughes and Shou-Cheng Zhang. Xiao-Liang Qi also wrote a review chapter on this topic, which you can partially access here on google books (but perhaps/hopefully there is a better source somewhere).
Disclaimer: Above I was a bit sloppy in referring to the effective Chern-Simons action as giving us a TQFT. It is true that we are looking at a quantum phase of matter, and we started with a partition function where we integrate over quantum-mechanical d.o.f. (the fermionic fields $\psi$). However, at the end we just have an action for an external gauge field $A$---the latter is not integrated over and thus not quantum-mechanical. Hence, in this case, the topological Chern-Simons action actually leads to a classical topological field theory, rather than a quantum-mechanical one. This is a general property of non-interacting fermionic topological phases of matter as well more generally of symmetry-protected topological phases of matter (see next example). These phases do not exhibit intrinsic topological order, which means their effective action is a classical one rather than a quantum-mechanical one (see our last example, namely that of intrinsic topological order, to see a genuine TQFT).
Disclaimer of the disclaimer: The current example under discussion---the Chern insulator---is actually a bit of an odd one out in the world of non-interacting topological phases. The reason for this is that unlike other topological insulators, it actually does not require the fine-tuning of symmetries to be stable. (Compare this to e.g. the Su-Schrieffer-Heeger chain discussed in J. Murray's post, which requires sublattice symmetry to have a well-defined topological invariant.) This means that it actually does admit a genuine TQFT description (despite not having genuine topological order); I will briefly come back to this point in the last section/example of this post. For this reason, the Chern insulator lives in the twilight zone between symmetry-protected topological phases and intrinsic topological order. Hence, it is not actually the best representative of the world of topological insulators, but it's just so nice that I could not resist discussing it.
Example: symmetry-protected topological phases of matter
Since my first and last example are quite long, let me attempt to keep this middle example brief. The main point is that topological insulators have a natural extension to the case of interactions, resulting in a concept known as 'symmetry-protected topological (SPT) phases of matter'. Like all topological phases, they can be characterized by an effective topological action describing their universal features.
SPT phases require the presence of symmetries in order to be non-trivial. This means that the fundamental degrees of freedom have some well-defined charges under a symmetry, which in turn implies we (usually) have a well-defined notion of how to couple these particles to an external gauge field for that symmetry. Similar to before, by integrating out the physical degrees of freedom, this then gives an effective partition function or action of the form $Z[A] = e^{iS_\textrm{eff}[A]}$. Here I work in real time, such that action is multiplied by the imaginary unit. Non-trivial SPT phases are characterized by non-trivial phase factors in this partition function.
As a concrete example, we can consider the Haldane phase which is realized in the spin-1 Heisenberg chain and deformations thereof. This is a non-trivial SPT phase in the presence of spin rotation symmetry $SO(3)$. In fact, one does not need the full spin-rotation symmetry: it is sufficient to preserve its $\mathbb Z_2 \times \mathbb Z_2$ subgroup generated by $\pi$-rotations around the three axes. As before, one can couple the spin chain to an external gauge field for these two $\mathbb Z_2$ symmetries and then integrate out the spins (in fact, one can do this calculation exactly for a clever choice of initial Hamiltonian, but I won't include this here since this post is already bursting at the seams). The result is a topological action/partition function $Z[A]$ on a spacetime torus (i.e., we imagine space and time both being a circle). This partition function has the special property that it is negative if one of the $\mathbb Z_2$ gauge fields has a non-trivial flux along the spatial direction and the other $\mathbb Z_2$ flux along the timelike direction. This negative sign serves as a topological invariant which distinguishes it from the trivial phase!
Example: intrinsic topological order
Now for the big kahuna: intrinsic topological order. This is the richest class of the three. In the examples discussed thus far, we had to couple the system to an external gauge field to get a non-trivial topological action. This is related to the fact that topological insulators and SPT phases have no interesting bulk excitations, i.e., there are no emergent anyons with unusual statistics and no emergent gauge fields. In contrast, phases of matter with intrinsic topological order give rise to these exotic features, which also means their low-energy topological action is non-trivial even in the absence of coupling it to external fields.
In the simplest types of topological order, the topological action is again the Chern-Simons action we saw above! (Fortunately recycling is a virtue.) However, it is now a genuine quantum theory: the gauge field is no longer a fixed external classical gauge field, but an 'internal' gauge field which we integrate over in our partition function:
$$ Z = \int e^{-S_\textrm{CS}[a]} \mathcal D[a]. $$
Two natural questions might be: (1) what does this internal gauge field $a$ refer to? And (2) what is the difference/consequence of having it be a dynamical gauge field that we now integrate over?
With regard to the first question: this internal gauge field $a$ refers to an emergent concept. It is seemingly not there in the underlying lattice model but only arises as an effective degree of freedom at low energies. It represents a collective motion, similar to how water waves represent the collective motion of billions of water molecules. One typically does not directly derive this emergent property. Instead: one writes down possible effective TQFTs, one analyzes their properties, and then one matches those properties with known properties of certain lattice models/phases of matter. Once these match, we declare that we have found the effective TQFT describing this phase of matter. In case this sounds haphazard: the universality classes describing intrinsic topological order have been classified and it is known what are the minimal set of properties one needs to check in order to claim one knows all of its universal properties.
Let us now address what the physical consequence is of integrating over our gauge field. It turns out to drastically alter the physics. In particular, for $k>1$ the physics is no longer that of Chern insulators (which is essentially integer quantum Hall physics) but instead that of fractional quantum Hall physics! For instance, the topological action implies the presence of fractionalized quasiparticles. I will not go into that here. Instead, I will show how this topological action implies that there is a topological ground state degeneracy which depends on the genus of the manifold. For concreteness, I will show how to derive that the ground state on a torus is $|k|$. (Here I largely follow the lecture notes by Sachdev.)
The Lagrangian density is
$$ \mathcal L = \frac{k}{4\pi} a \mathrm d a = \frac{k}{4\pi} \left( a_t \partial_x a_y - a_t \partial_y a_x + a_x \partial_y a_t - a_x \partial_t a_y + a_y \partial_t a_x - a_y \partial_x a_t \right). $$
If we are on a closed spatial manifold (or even on a manifold with boundary where we choose the gauge $a_t=0$ at the boundary), then by partial integration we can write
$$ \mathcal L = \frac{k}{2\pi} a_t \underbrace{\left( \partial_x a_y - \partial_y a_x\right)}_{=\textrm{ magnetic field}} + \frac{k}{4\pi}\left( a_y \partial_t a_x - a_x \partial_t a_y \right). $$
Note that the only place where $a_t$ appears is in front of the magnetic field; hence, it functions as a Lagrangian multiplier such that integrating over $a_t$ effectively pins us to gauge fields where the field strength is zero (we say they are flat):
$$ Z = \int e^{-\frac{k}{2\pi} \int a_t \left( \partial_x a_y - \partial_y a_x\right) \mathrm dt \mathrm d x \mathrm dy} \mathcal D[a_t] e^{ -\frac{k}{4\pi} \int \left( a_y \dot a_x - a_x \dot a_y \right) \mathrm d t \mathrm dx \mathrm d y} \mathcal D[a_x,a_y] = \int e^{ -\frac{k}{4\pi} \int\left( a_y \dot a_x - a_x \dot a_y \right) \mathrm d t \mathrm dx \mathrm d y} \mathcal D[a_x,a_y]_\textrm{flat}.$$
We should thus only integrate over gauge fields with zero field strength. Fortunately, we can do this by fixing our gauge such that we only have gauge fields which are constant in space. Indeed, to see this, let $a_x(x,y)$ and $a_y(x,y)$ be a general position-dependent gauge field. Now define the function $\lambda(x,y) = \int_{(0,0)}^{(x,y)} \vec a \cdot \mathrm d \vec r$. Note that the flatness condition of the gauge field implies that this function is well-defined, i.e., it is path-independent. It is then easy to confirm that the gauge transformation $a_i \to a_i - \partial_i \lambda$ (for $i=x,y$) makes the gauge field constant (i.e., in this gauge we have that $\partial_i a_j = 0$). Hence, instead of a path integral for fields defined on space, our partition function is now a simpler object where we have two scalars which depend on time
$$ Z = \int e^{-\frac{k}{4\pi} \int \left( a_y \dot a_x - a_x \dot a_y \right) \mathrm dt \;\times \iint \mathrm d x \mathrm d y} D[a_x(t)] D[a_y(t)] = \int e^{-\frac{k}{4\pi} \int \left( b_y \dot b_x - b_x \dot b_y \right) \mathrm dt } D[b_x(t)] D[b_y(t)],$$
where in the last equation we introduced the rescaled fields $b_i = a_i \times L_i$ for convenience (where $L_{x,y}$ is the length in the two directions of the torus).
Hence, the resulting Lagrangian (after partial integration on a closed manifold) is
$$ L= \frac{k}{2\pi} b_y \dot b_x .$$
Note that the momentum conjugate to $b_x$ is $\Pi_x := \frac{\delta L}{\delta \dot b_x} = \frac{k}{2\pi} b_y$. Hence, the gauge fields $b_x$ and $b_y$ are conjugate with the commutation relation $[\hat b_x,\hat \Pi_x] = i$, i.e.:
$$ [\hat b_x,\hat b_y] = \frac{2\pi i}{k} .$$
Also, observe that the Hamiltonian is trivial: $H = \Pi_x b_x - L = 0$. This is typical of TQFTs, since the theory has no local content (and energy is local). This means that any operator that is well-defined in our theory is a zero-energy operator, in the sense that it maps zero-energy states to zero-energy states. Remarkably, our theory has non-trivial operators: defining the Wilson loop operators that wrap around the torus, i.e. $\hat W_x = e^{i\int_0^{L_x} \hat a_x \mathrm d x} = e^{i \hat b_x}$ and $\hat W_y = e^{i \hat b_y}$, the aforementioned commutation relations imply
$$ \hat W_x \hat W_y = e^{2\pi i/k} \hat W_y \hat W_x . $$
Since these are zero-energy operators that do not commute, it follows that the ground state manifold is degenerate. Indeed, suppose $|\psi_0\rangle$ is a ground state which is an eigenstate of $\hat W_x$, then $|\psi_n \rangle := \hat W_y^n |\psi_0\rangle$ are also ground states for any choice of $n\in \mathbb Z$, but due to the aforementioned commutation relations, the eigenvalues of these states under $\hat W_x$ depend on $n \mod k$ and we thus have $k$ linearly independent ground states on the torus! QED
The easiest way to go beyond this simple Chern-Simons action is by including multiple emergent gauge fields. E.g., $\mathbb Z_2$ topological order (which I mentioned toward the beginning of this post) is described by a mutual Chern-Simons action: $S[a,b] = \frac{1}{2\pi} \left( a \mathrm db + b\mathrm d a\right)$, where we integrate over both $a$ and $b$ in the partition function. More complicated topological orders will require more complicated topological actions to describe them. For those interested in further reading, the early seminal paper by X.G. Wen has an illuminating discussion on these actions.
Extra tidbit: We see that the case $k=1$ above does not lead to a degeneracy. Indeed, this special case does not have intrinsic topological order. In fact, it is a different description of the Chern insulator we described in our first example! To see this, we first need to include a coupling to an external gauge field: $S[a,A] = \frac{1}{4\pi} a \mathrm d a + \frac{1}{2\pi} a \mathrm d A$. The partition function is then:
$$ Z[A] = \int e^{-S[a,A]} \mathcal D[a] = \int e^{-\frac{1}{4\pi} \int (a+A) \mathrm d (a+A) + \frac{1}{4\pi} \int A \mathrm d A} \mathcal D[a] = e^{ \frac{1}{4\pi} A \mathrm d A} \times \int e^{-\frac{1}{4\pi} \int b\mathrm d b} \mathcal D[b] $$
where $b = a+A$ is a change of variables. We indeed recover the topological action for the external gauge field $A$, agreeing with what we saw for the Chern insulator with Chern number $k=1$. To get a higher Chern number, we just need to include multiple (decoupled) internal gauge fields, each described by the Chern-Simons TQFT at level $k=1$.