Why isn't the graph a straight line but is rather curved ?
The graph you posted is of the barometric formula. Compared to the link I posted, your axes are swapped, but nevermind. It describes the exponential decay of atmospheric pressure with increasing altitude. As exponential functions are curved (in a linear axis scaling as yours), your graph is not a streight line. The physics behind this comes from the fact, that air is a compressible gas. Air in a certain altitude is compressed by the air above. If you climb up a little, there is less air above you. This leads to the air being less compressed, meaing lower pressure and also meaning its less heavy per volume.
If you follow your graph from the righthand side (1000 hPa at sea level), and travel to the left of your figure, you'll notice that you never hit the y-axis (which would be 0 hPa). If your graph was a streight line, you'd inevitably hit zero hPa. This would then be the sharp "upper border" of the atmosphere. But this does not exist. Atmosphere (and therefore air pressure) gets thinner and thinner, and does not abruptly end at zero pressure. That's why the international space station (ISS) - while being "in space" - still experiences a little air drag while circling around earth.
If you sat yourself on the bottom of the ocean, you'd experience a lot of water pressure, as you and the surrounding water are pressurized by weight of the water above you. Water is - in contrast to air - not a "compressible" fluid, meaning it has the same weight per volume, regardless of pressure. (Well this is not 100% correct. But at least 99.9% correct. Which is good enough for now.) Now, if you start to swim upwards, the pressure is becoming less, but water's density does not change. This would lead to a graph with a streight line. If you continue upwards - as mentioned in the previous example - you will eventually hit the "ceiling" of the fluid, as the pressure becomes zero. This is the surface of the water.
So, why is the graph curved? Because atmosphere doesn't have a upper surface. :-)