Topological Insulators: is HgX a special case? I got confused by reading this article:

Francois Virot, Roland Hayn, Manuel Richter, and Jeroen van den Brink. “Engineering topological surface-states: HgS, HgSe and HgTe.” arXiv preprint arXiv:1306.0999 (2013).

The article says that topological surface states include contibutions from dangling bonds. And later, passivating the dangling bonds, they actually open a gap in Dirac cones present in surface.

I get confused whether this materials are classified as TI just because they are insulators in their bulk and conductors in surface or are them TI as any other.
So, my questions are:


*

*Are there two definitions of TIs, one for usual TIs, and another (wider) including HgX?

*Are these (HgX) topological insulators a result of strong spin-orbit coupling and time-reversal symmetry?

*Weren't surface states of a TI topologically protected? Has HgS stopped being a TI by passivating its dangling bonds? (a gap has appeared)
I've found these related topics, but still unclear:


*

*Would HgTe be a topological insulator?

*How Fundamental is Spin-Orbit Coupling to Topological Insulators?

*Is edge state of topological insulator really robust？
 A: 
1. Are there two definitions of TIs, one for usual TIs, and another (wider) including HgX?

No, there is only one definition a 3D band topological insulator (TI): the $\mathbb{Z}_{2}$ classification scheme proposed by Fu, Kane, and Mele:

Liang Fu, Charles L. Kane, and Eugene J. Mele. “Topological insulators in three dimensions.” Physical Review Letters 98, no. 10 (2007): 106803. (arXiv)

In the literature on 3D TIs people often refer to these so-called “strong” and “weak” TIs. This can potentially be misleading. Superficially, it gives the impression that there are two types of topologically nontrivial insulating states; if that were the case then the classification wouldn't be $\mathbb{Z}_{2}$. The strong TI is the only “real” TI with protected surface states. A weak TI is something which you can think of as a stack of 2D TIs. But a weak TI is topologically trivial; hence not a real TI. The surface states of a weak TI can be destroyed by things like surface passivation, doping, pressure, etc. (without closing the bulk gap). I know your question is not about the distinction between strong and weak TIs, but since this was so closely related, I thought I'd mention it.
Anyways, getting back to your question, the above paper says that $\beta$-HgS was predicted to be a strong TI according to Ref. 21. So we don't need to worry about surface passivation destroying the surface states. The gap that you are seeing in the rightmost plot in Fig. 2 is simply due to the overlap of the surface states on the opposite surfaces of the slab. The authors used a 12 unit cell thick slab of HgS for their analysis; the authors claim they can't go any higher than 12. If they make the slab thicker the gap will disappear. You can note that they say in the text:

“For the (110) surface, where less dangling bonds exists than at (001), we find an intermediate situation with weight maximum at the surface but about 2 nm decay length.”

Since the surface states penetrate deeper into the bulk after surface passivation (2 nm instead of 1 nm), it's easier for the states on the opposite surfaces of the slab to overlap; the larger the overlap the larger the gap. The authors did increase (from 8 to 12) the thickness of the slab by 4 unit cells after passivation. Evidently, that was not enough.
As a matter of fact, there is a very tiny gap (with 9 unit cell slab) in the Dirac cone even in the leftmost plot of Fig. 2. The authors even mention the size of this gap: 2.7 meV. Once again, this (barely visible) gap on the (110) surface is also a result of surface state overlap on opposite surfaces of the slab. In conclusion, the gap that you see in the Dirac cone on the passivated (001) surface of HgS is nothing to worry about; it is simply a result of computational limitations.

2. Are these (HgX) topological insulators a result of strong spin-orbit coupling and time-reversal symmetry?

Yes, they are. As I explained above, there is only one type of topological insulator. So “these” topological insulators (HgX) inherit the same properties as (say) Bi$_{1-x}$Sb$_{x}$, Bi$_{2}$Se$_{3}$, etc.

3. Weren't surface states of a TI topologically protected? Has HgS stopped being a TI by passivating its dangling bonds? (a gap has appeared)

No, HgS is a TI even after passivation. It would not (by definition) be a TI otherwise. For the reasons I stated above, the gap you see in that plot is just an artifact. The gap will not occur in the Dirac cone (passivated or not) if we had the surface of a macroscopic HgS (not a slab).
Disclaimer: I have not gone through the papers claiming that HgS is a strong TI. I just took it for granted. However, my comments above are not limited to HgS.
