This potential is a Lennard-Jones potential which is an approximate model of interatomic potential functions that captures the qualitative behavior. The asymmetry is because the physics causing the left part of the potential (steep, repulsive) is different from the physics causing the right part of the potential (less steep, attractive). Paraphrased from Wikipedia:
The repulsive region (left part of the potential) describes Pauli repulsion at short ranges due to overlapping electron orbitals, and the attractive long-range region (right part of the potential), describes attraction at long ranges (van der Waals force, or dispersion force)
Visualize this like old school paddle ball toy. When the atoms get really close together (ball hits the paddle), there is a very sudden and very strong repulsive force that pushes them apart. However, as the atoms get farther apart (ball moves away from the paddle and stretches out the elastic string) there is a weaker attractive force that slowly pulls them back together again. The analogy breaks down because for atoms, the attractive force is weaker the farther apart you separate them, but hopefully you get the idea of "strong, sharp repulsion" versus "weak, gradual attraction."
If a solid is at a low temperature and the atoms have low energy, it is stuck near the bottom of that potential well, which is more symmetric, and it just rolls back and forth. As you heat the solid and the atoms gain more energy, they can roll up higher in that potential well.
Imagine that potential function were a literal (frictionless) hill in that exact shape. You release a marble from the right edge, just lower than the highest point at the right. That marble will slowly start to roll down in toward the bottom, picking up speed gradually. It will then hit the bottom at high speed and shot up the steep wall to the left, but then come right back down very fast because the wall is so steep. Then it will roll off to the right again and slowly lose speed until it gradually comes to a stop, turns around, and begins the process again. On average (with respect to time) the marble spends more time on the right part of the potential, where it is farther away, than on the left part, where it is closer. This is just because the right side is less steep. Likewise, the atoms spend more time farther apart because the (attractive) restoring force is weaker for separation than it is for repulsion. If the atoms on average are spending more time farther apart, then the solid as a whole is on average larger, because its atoms are more separated