Why does orbital overlap cause attraction? I have been taught that when orbitals overlap as in $\sigma$ and $π$ bonds, the formation of a bond (which is basically an attraction) takes place. Why does this cause attraction, shouldn't they repel? I get that electrons get paired up because they have opposite spins which creates magnetic attraction, so is this the same thing happening here?
If yes, then please elaborate, if not, then also please elaborate.
 A: In this "overlap" explanation, the overlap is not an overlap between two electrons. You get a $\sigma$ bond in the H2+ molecule, where there is only one electron.
There are two reasons why you get a lower energy in a covalent bond. One is that the kinetic energy is lower than in the unbound state, and the other is that the potential energy is lower. The overlap explanation relates to the kinetic energy.
In a covalent bond, the electron is spread out between two different atoms. This spreading means that it has a longer wavelength, which corresponds to a smaller momentum and smaller kinetic energy.
The overlap idea comes in only because the electron can't spread out into the two atoms if the two orbitals that the electron is simultaneously occupying have a lot of empty space between them.
A: When you put two atoms close enough together that both nuclei are important to the behavior of both sets of electrons, you have significantly changed the potential that an individual electron is moving within. This potential generates wavefunctions that are quite different from the wavefunctions in an isolated atom, with different energies and very different wavefunction shapes.
The point is: when two atoms interact nontrivially, their old wavefunctions don't simply add together. The single-atom wavefunctions simply aren't valid anymore*, and the ones that exist in the molecule look quite different.
When we solve the Schrodinger equation for a two-point-charge potential, it turns out that, in general, we get two classes of orbitals: bonding orbitals, in which the electron wavefunction is concentrated in the center of both nuclei, and antibonding orbitals, in which the electron wavefunction is concentrated at the outer edges of the molecule (in fact, the antibonding wavefunction actually has a node, a location with zero amplitude, between the two nuclei). 
In the bonding wavefunction, the electron is concentrated in a region where the potential energy is lower than it is for a single atom (because it's close to two nuclei at once); in contrast, an electron in the antibonding orbital has a higher potential energy than an electron in a single atom (because it's, on average, further away from a nucleus than it was for a single atom). 
The electrons from both atoms' outer shells fill these molecular orbitals, starting with the ones at lowest energy. The more electrons in a bonding orbital there are, the stabler the molecule will be; however, populating the antibonding orbital generally makes the molecule less stable. This constrains the number of electrons that can be involved in a bond.
There's a whole discipline of knowledge called molecular orbital theory that is entirely about what happens in these kinds of cases. Acquainting yourself with the basics of this field will likely answer most of your questions.

*For multielectron atoms this only really applies to electrons in outer shells, and only approximately at that. Electrons in inner shells are much closer to their respective nuclei, and so the distortion to their wavefunction from the other nucleus can still be safely neglected in the first approximation.
A: You are asking why the formation of covalent bonds between two (or more) atoms causes attraction (bonding) between them.
Now it is very important to understand that the more correct expression is the sharing of electron orbitals that causes this attraction (overlap is not even needed, when there is only one electron being shared).
The sharing (between the atoms) of electron orbital causes the electron probability density to become very significant on the nuclear axis, between both nuclei and causes the intra-nuclear Coulombic repulsion between the atoms to reduce, and this is what we  could interpret as attraction (covalent bonding).

To the right is also schematised the electron probability density ψ2 and note that this density is very significant on the nuclear axis, between both nuclei. This causes the Coulombic repulsion force between the nuclei to greatly reduce and the molecular arrangement to be stable, meaning that pulling it apart would cost energy. This energy is often referred to as the bond strength

What gives covalent bond its strength?
A: I am answering with these quotes(and also  my comment to probably_someone)

Since opposite charges attract via a basic electromagnetic force, the negatively-charged electrons orbiting the nucleus and the positively-charged protons in the nucleus attract each other. Also, an electron positioned between two nuclei will be attracted to both of them. Thus, the most stable configuration of nuclei and electrons is one in which the electrons spend more time between nuclei, than anywhere else in space. These electrons cause the nuclei to be attracted to each other, and this attraction results in the bond. However, this assembly cannot collapse to a size dictated by the volumes of these individual particles. Due to the matter wave nature of electrons and their smaller mass, they occupy a very much larger amount of volume compared with the nuclei, and this volume occupied by the electrons keeps the atomic nuclei relatively far apart, as compared with the size of the nuclei themselves.

....

In the simplest view of a so-called covalent bond, one or more electrons (often a pair of electrons) are drawn into the space between the two atomic nuclei. Here the negatively charged electrons are attracted to the positive charges of both nuclei, instead of just their own. This overcomes the repulsion between the two positively charged nuclei of the two atoms, and so this overwhelming attraction holds the two nuclei in a fixed configuration of equilibrium, even though they will still vibrate at equilibrium position. In summary, covalent bonding involves sharing of electrons in which the positively charged nuclei of two or more atoms simultaneously attract the negatively charged electrons that are being shared. In a polar covalent bond, one or more electrons are unequally shared between two nuclei.

Of course as the whole is a quantum mechanical system the electrons ( and nuclei to a much much smaller in space volume) are described by orbitals, i.e. the quantum mechanical probability functions for their existence at a particular (x,y,z,t).  The shape and directions of the electron orbitals may very well be as described in the other answers, but the basic attraction is between the negative electrons and the positive nuclei.
