How high can be tower or building? I tried to find in the internet some scientific explanation and calculation and looks like it is difficult. I found some calculation for house from standard bricks and it gives $170~\mathrm{m}$ for Ultimate tensile strength of brick $3~\mathrm{MPa}$. Formula is 
$$
h = \dfrac{\sigma}{\rho g}
$$
where $\sigma$ is Ultimate tensile strength.
this formula is explained here
As I see here there is kind of steel with Ultimate tensile strength $2600~\mathrm{MPa}$ and according to this formula it can be 32 km!
As I understand this formula for square or cylindrical shape, like the same width everywhere. But what if we make it in the shape of Eiffel Tower or kind of hyperbolic and etc? Or shape is does not matter and maximum height will be the same?
Or maybe if we build it from sticks with the same shape as diamond crystal structure we can build it up to 100 km?
Is there any well know way (formula) to calculate maximum height of the tower of the complicated shapes?
UPDATE:
As I was told, I should use Compressive strength instead of Ultimate tensile strength. It looks reasonable. In this case calculation will be the same, only for steel I found value not 2600 MPa, but 300 MPa, but I can take another material from here with the similar value 2600. and if I take diamond with Compressive Strength 17000 MPa it will give 480 km.
UPDATE2/ANSWER:
Looks like I found answer by myself with help of all your valuable comments. If I use assumptions like Total gravitational force to the basement less or equals breaking force ($\sigma$S) where $\sigma$ - compressive strength and S - area in square meters, I get this formula for cylinder
$$
h \leq \dfrac{\sigma}{\rho g}
$$
some numbers for cylinder:
Steel (300 MPa): 3.75 km
Granite (300 MPa, but less density than steel): 11.5 km
Diamond (17000 MPa): 480 km
ABS Plastic (65 MPa): 6.5 km
Strongest concrete (80 MPa): 3.2 km
Carbon epoxy (up to 1500 MPa): 100 km
but for real building we have to divide it to 2 or 3 to have some "factor of safety". In this case only diamond and carbon epoxy can be used.
For cone
$$
h \leq \dfrac{1}{3} \dfrac{\sigma}{\rho g}
$$
numbers will be 3 time more than for cylinder.
For other shape this condition should be met
$$
\sigma \geq \dfrac{\rho g V}{S}
$$
I tried to calculate Exponential cone like this

which is Solid of revolution of this function
$$
f(x) = r e^{-α x}
$$
where r is basement radius and α is kind of cone steepness. Volume can be calculated by formula from wikipedia article Solid of revolution
Looks like for this exponential cone it is possible to build tower of any high from any material, but for materials with low compressive strength, if we take basement r=1 km, desired height 100 km for example, last 70% of the tower it will be very thick (like $10^{-5}$ meters). Of course this kind of needle is not possible to build and it does not make any sense to build. if we accept that final radius at maximum height 100 km equals 0.5 meter, the basement radius for different materials will be like this.
r without "factor of safety"
Steel (300 MPa): 400 km
Granite (300 MPa, but less density): 42 m
Diamond (17000 MPa): 0.56 m with "factor of safety" 5
ABS Plastic (65 MPa): 1.5 km
Strongest concrete (80 MPa): 5000 km
Carbon epoxy (up to 1500 MPa): 0.51 m
if we think about "factor of safety" equals 3, as I understand it is standard for this kind of things, we get this numbers
Steel (300 MPa): $10^{14}$ km
Granite (300 MPa, but less density): 230 km
Diamond (17000 MPa): 0.56 m with "factor of safety" 5
ABS Plastic (65 MPa): 5500 000 km
Strongest concrete (80 MPa): $10^{17}$ km
Carbon epoxy (up to 1500 MPa): 2 m
In this case we can really build a space elevator with high 100 km from 3 materials Granite, Diamond, Carbon epoxy. Even yearly production of Carbon epoxy will be enough to build it :)
This is exact picture for tower with 100 km height from carbon epoxy and factor of safety 3 (all axes in meters)

Of course I do not consider wind, and all other things. With precise engineering calculation might be it will not be possible.
 A: If you make a pile of bricks, you need to talk about strength in compression, not tension. Bricks and stone are much stronger in compression than tension. 
This had a strong influence on architecture. The Parthenon is a lot of closely spaced columns because you can't lay a stone over a long gap. The bottom is under tension and fails. 
When the arch was discovered, it became possible to open up the interior of buildings. An arch is under compression over its entire length. Medieval cathedrals took advantage of this. 
Modern materials make taller buildings possible. Steel is strong in tension. More designs become possible. Wind loads and earthquakes matter. They try to push a building over. The tallest buildings are stretching the limits of materials.
Mountains are limited by the strength of the crust of the Earth. A mountain that is too tall will slowly be shortened as the crust sags. 
The crust is moving, so it isn't as simple as that. Mt. Everest is still rising as the Indian subcontinent rams into Asia. 
The tallest mountain is the island of Hawaii. It is still growing because it is an active volcano. 
I don't know that either of these push the limits of crust. Mountains may not get big enough to do that because nothing causes them to grow without limit, or because they erode about as fast as they grow. 
But there are areas in Scandinavia that used to be under glaciers that are rising. 
