First of all, as mentioned, atmospheric pressure can exert very high loads when integrated over significant areas. As an example, an overpressure of just 2psi is sufficient to destroy many houses and can kill people. That's about 13% of atmospheric pressure.
Secondly there is an important scale question. You give an example of a bike tyre: a road bike tyre is often inflated to 8 bar or more: if you inflated a car tyre to 8 bar ... well, don't. The reason for this is that the tension in the structure of the tyre goes like $\mathrm{pressure}\times\mathrm{radius}$: so larger tyres have linearly more tension for the same pressure, and would therefore have to have linearly thicker walls to withstand the same pressure.
Finally and most importantly there is an enormous difference between the behaviour of structures (as opposed to materials) under compression and under tension. This is really a huge engineering area but even physicists can see why it's true. Consider for instance a steel rod 2mm in diameter and a metre
long: you could easily hang from such a rod, because it's very strong in tension. But if you tried to stand on it then it would collapse immediately. This happens because when the rod is in tension it's in stable equilibrium -- if it bends a bit then the tension pulls it straight -- while if it's in compression then the equilibrium is much less stable -- if it bends a bit then there is an enormous lever which causes it to bend further and it can abruptly collapse. The details of this are complicated and engineers spend a lot of time working out how to design structures which are strong in compression and bending and what the stresses are in them, but the principles are easy to understand.
So tyres, for instance, are nearly pure tension structures: a tyre would not support even a tiny positive pressure outside it. Designing structures which work in compression, such as submarine hulls, is really hard, and they are vulnerable to catastrophic collapse when their design strength is exceeded. Similarly a railway car is designed to support a (small) internal pressure which places its structure in tension, but when it is in compression it will collapse immediately.