Why is it convenient to define zero electric potential at infinity? I get that electric potential is defined at infinity for convenience but why is it convenient? What would be an inconvenient place to define zero electric potential?
 A: The electric field at position $\mathbf r$ of a point charge $q$ located at the origin is given by
$$\mathbf E=\frac{kq}{r^2}\hat r$$
By definition, the electric potential at some position $\mathbf r$ due to this configuration is then
$$V=-\int_\mathcal{O}^\mathbf{r}\mathbf E\cdot\text d\mathbf l$$
where $\mathcal{O}$ is your "reference position" where $V=0$. So, let's pick a general reference position $\mathbf r_0$ and find the potential relative to it. Due to the field being conservative, any path from $\mathbf r_0$ to $\mathbf r$ will give the same result. Due to the radial symmetry we can just move radially from radius $r_0$ to radius $r$ and then move along the final circle to the final position $\mathbf r$
$$V=-\int_{r_0}^r\frac{kq}{r'^2}\text dr'=\frac{kq}{r}-\frac{kq}{r_0}$$
So, as you can see, for any finite $r_0$ we have this extra term of $-kq/r_0$ floating around. However, if we choose $r_0\to\infty$, then this term vanishes, leaving behind only one term $V=kq/r$.
Note that this only matters if you want to talk about some sort of "absolute potential" relative to $\mathcal O$. If you are looking at potential differences then the reference point becomes irrelevant.
Also note that a similar analysis can be done on other charge configurations as well. For ones where the configuration goes out to infinity itself we can no longer pick infinity as a reference point.
A: 
I get that electric potential is defined at infinity for convenience
but why is it convenient?

While it may be convenient and logical in the case of a point charge, it may not be convenient, or even necessary, in other cases, particularly in the case of circuit analysis.
For example, if we needed to determine the potentials at various points in an electric circuit in order to apply Kirchhoff's current law in nodal analysis, assigning zero potential at infinity would not be convenient. Instead, it is convenient to assign zero potential to some point in the circuit, typically a point that is common to several branches, a "grounding" connection, or the negative terminal of a voltage source. In applying Kirchhoff's voltage law in loop analysis we are only interested in potential differences, not absolute potentials.
Hope this helps.
A: It's convenient in that way for a point charge to having zero potential at infinity since the electric field dips with increasing distance.
For charge distributions however, we can have a zero potential at some finite distance. Here's one example:
Say we had a cylindrical charge distribution, where the electric potential is of the form $V \propto ln(r/r_0) $ where $r$ is some distance from the charge source and $r_0$ is an arbitrary distance we set up. If we calculate the new zero point, we would see this to be at $r = r_0$. Moreover, this doesn't impact much physics by assuming such an arbitrary distance, since we're mostly always interested in difference in potential.
A: 
... but why is it convenient?

Of course you could define it to be zero at a certain distance (e.g. 1m or 1ft) from the center of charge.
However, in this case the definition becomes more complex because it would depend on the definition of the unit needed for the distance:
If you would define that the potential is zero in a distance of one mile, your definition would depend on the definition of the unit "mile". And does "mile" mean nautical mile (1852m), American land mile (1609m) or even historic German miles (7532m) ... ?
There are two distances where this problem does not occur:
Zero and infinite distance.
Zero distance is not possible because zero distance would mean infinite energy so the only possible definition that does not depend on a length unit is infinite distance.
